Facebook ads ‘Interest Categories’ may be up to 33% inaccurate

Experiment 1. The first experiment involved setup of 14 accounts, a small sample size, that merely viewed or scrolled through a page in order to see if those topics within the content consumed would be pulled into the “Interest Categories” accounts. The goal was to see what interests would then be associated with the newly formed account and to qualitatively infer how accurate the newly assigned interests were.

The findings show 33.22% of the inferred interests were either inaccurate or irrelevant. “The key finding here is that Facebook takes an aggressive approach to interest inference,” Aafaq Sabir, lead author of a paper on the work and a Ph.D. student at NC State, said.

Experiment 2. To get even deeper, the NC State team wanted to see if the findings would hold true for a more diverse group of users. The 146 study participants were selected by Amazon Mechanical Turk from “different parts of the world”. A browser extension would then extract data from the participants’ Facebook accounts and question the participant about the validity of the interest data.

This study found 29.3% of the interests that Facebook had listed for participants were not actually of interest. “That’s comparable to what we saw in our controlled experiments,” said Anupam Das, co-author of the paper and an assistant professor of computer science at NC State.

What’s unclear. With samples sizes this small, we should take this study with a grain of salt. While the data is unflattering to Facebook’s targeting, much of the experiment is unclear on the parameters used to determine what is of interest and what is not of interest. Additionally, the second experiment in this study relies on user feedback that does not appear to have quantitative parameters in place for reporting, which may muddy results. Lastly, the participants selected in the survey were in different locations, but sourced similarly using a very specific platform – Amazon Mechanical Turk. By choosing participants tied to one source, it may not be a truly accurate representation of Facebook users as a whole.

Why we care. Most ad targeting isn’t perfect. For example, if logged in, you can see what Google targets your ads to and see what Facebook thinks that you are interested in. The reality is that no targeting is perfect, but the 30-33% inaccuracy found in the study is concerning. Before pulling ad spends to different platforms, it is worthwhile to assess the current account performance and make adjustments from that data instead of a study. Finally, the first portion of this study does show that basic content consumption can influence interests, so again check your results to ensure that your targeting is in fact working for you.