IMDb Review-bombing: from Ghostbusters (2016) to She-Hulk (2022)
Every year, a few movies (and TV shows) garner an unusually high volume of terrible ratings on IMDb. The first time such “review bombing” caught my attention was in 2016 with the release of the female-led Ghostbusters (2016) reboot. The ratings were so irregular that IMDb decided to employ an “an alternate weighting calculation” and it now has a solid 6.9/10.
More recently, Marvel’s latest She-Hulk: Attorney at Law suffers from 37.5% of all its ratings being 1/10s (as of September 14, 2022). A back-and-forth debate of “this screams misogyny” and “well why don’t Wonder Woman and Kill Bill also have low ratings then?” plagued my social media. Though we can always come up with women-starring movies appreciated by IMDb users, it does not mean that the attacks on She-Hulk and Ghostbusters are not founded on misogyny. A lot of misogynists love women - as long as they fit in the box of what a woman should be and do. Maybe She-Hulk and Ghostbusters are outside of that box, or maybe they truly are terrible.
Question of interest
The goal of this post is to use my amateur web scraping and more-than-amateur data analysis skills to explore the following question: are specific kinds of movies more likely to encounter hate on IMDb than others? Are movies with a woman lead more likely to be hated on than movies with a non-woman lead? How about if the two leads are women? If it was (co-)written by a woman, or (co-)directed by a woman? I’ll also look into non-gender variables obtainable on IMDb. The full list of variables considered, and their definitions, is displayed in the Glossary.
How do we measure hate? What constitutes a hated movie?
By my choosing, a movie’s hate will be measured by the proportion of IMDb ratings that are 1/10 . The more 1/10 ratings a movie gets, the more hated it is.
A movie will be considered to be ‘Hated’ if it is one of the 100 most hated films in its Metascore group, ie if it is among the 100 films with the highest proportion of 1/10 IMDb ratings. Classifying movies into Hated or Not Hated was a personal choice, and so was the way a Hated movie is defined. I like this approach as it allows us to look at each kind of movie (eg woman-led, Horror, etc) and find the proportion of them that are Hated, giving us a naive estimate for the probability of hating a specific kind of movie. We can then compare that to the proportion of the other movies (eg non-woman-led, non-Horror, etc) that are Hated. If the difference is large enough for us, then we can conclude IMDb users are biased against/in favour of those kinds of movies.
Grouping movies by Metascore Rating
Now, a movie with a high proportion of 1/10s could simply be a terrible movie (as the “What about Kill Bill and Wonder Woman” crowd would argue). My proposed solution to differentiate the terrible movies from the not terrible movies is by separating films based on their Metacritic’s Metascore Ratings. Metacritic splits movies into three tiers: 1. Metascore between 61-100, 2. Metascore between 40-60 and 3. Metascore between 1-39. From hereafter, the 3 groups will be referred as Favourable, Mixed, and Unfavourable movies, respectively for brevity’s sake. Since movies are grouped within similarly-rated films, those with more 1/10 ratings can be interpreted as more hated rather than worse in quality.
Why Metascore?
Don’t care? Skip right to Films included in analysis.Why you might not like Metascore:
1. Critics are pretentious and do not represent the general population; they probably like art house films more than the average moviegoer, and blockbuster films less.
2. Any IMDb bias detected in this analysis can instead be attributed to Metascore bias in the opposite direction. That is, if we find a disproportionate amount of IMDb “hate” for woman-led films, maybe it’s actually a disproportionate amount of Metacritics “love” we are observing.
1. they are posted within movies’ IMDb pages so it makes scraping the data easier for me, and
2. they are independent of IMDb ratings since they are based purely on critic reviews, and
3. critics risk their professional reputation if they let their own personal biases turn their review into an exaggerated, take-down piece.
In my opinion, the pros outweigh the cons. I believe that a Metacritic-liked film with a high proportion of 1/10 ratings on IMDb is likely a victim of unjust review bombing.
Maybe you’re still not convinced. If it helps, below is a chart of the Metascore ratings of the 2400+ movies released between 2000 and 2019 that received 50K+ IMDb ratings; please feel free to hover over movies with high, medium, and low Metascores. Or in the table below (Table 1), you can look up the title of any movie in the Search bar to see its Metascore rating.
Films included in analysis
Don’t care? Skip right to Results.
Only movies released in the last two decades (2000-2019) will be included in this analysis. Movies with fewer than 50,000 ratings on IMDb are removed so that the films we investigate are more likely to be known by the average moviegoer (and reader of this post). Finally, we also removed movies not rated by Metascore. 2,417 movies on IMDb fit this criteria.
The pie chart below breaks down the 2,417 movies movies into the 3 Metascore-based groups. 44% (1062) are Favourable, 40% (972) are Mixed, and the remaining 16% (383) are Unfavourable.
How hated are the Hated movies?
The top 100 hated films for each group are determined. The proportion of 1/10 IMDb ratings for Hated movies ranges from
i) 2.6% to 11.2% for Favourable films,
ii) 4.0% to 46.4% for Mixed films, and
iii) 4.9% to 60.8% for Unfavourable films.
Results
- Horror: Favourable Horror films are 3.6x as likely to be hated as Favourable non-Horror films. This stark difference is also observed in Mixed movies.
- Women: For Favourable movies, Woman-led films are 1.9x more likely to be hated than non-Woman-led films. For Mixed movies, the gender effect is even stronger: films with a Woman Lead, 2 Women Leads, and a Woman writer are 2.1x, 2.4x, and 1.7x as likely to be hated than their corresponding counterparts, respectively.
- Auteur: Across all three Metascore groups, Auteur movies are at least 1.6x more likely to be hated than non-Auteur movies.
- Unfavourable films show the least correlation between IMDb hate and any of the movie traits. This supports the idea that critically disliked films earn their 1/10 ratings on IMDb; the hate is based on actual movie quality rather than any prejudice.
- Mixed films show the most correlations between IMDb hate and the considered movie traits. Perhaps when a movie is not great - as is the case for Mixed movies - it enables audience’s biases and leads to harsher, more exaggerated ratings.
The bar plot above shows the different proportions of hated movies for each movie trait, within each Metacritic group. The dark bars represent movies that have the given trait, and light bars do not have it. For instance, the dark green bar at the very top right represents the proportion of Favourable horror movies that are Hated (28.05%), and the light green bar to the very left of it is the proportion of Favourable non-horror movies that are Hated (7.86%). Note: movie traits representing fewer than 30 movies in our sample are not shown.
Summary
Gender biases
Critically acclaimed movies are way more likely to be hated when the lead actor is a woman than when they’re not a woman. Movies that have mixed reviews from critics, ie that are not great but not terrible, reveal even more gender-based biases; they are more likely to be hated when i) the lead actor is a woman, ii) the two lead actors are women, and also when iiii) one of the writers is a woman.
Two movies really punctuate the analysis for me: Entourage (2015) vs Sex and the City (2008). Both are spin-off films of their respective highly successful TV shows of the same name; one stars all men and the other all women. They are so comparable that Entourage is often described as “Sex and the City for guys”. Since the films’ releases, Entourage was critically disliked and grossed ~ 1.6x its budget whereas Sex and the City had mixed reviews, grossed ~ 6x its budget, and got a sequel film. And yet, despite the latter being more critically AND commercially liked, it suffers from an abysmal 10.6% of its IMDb ratings being 1/10. In contrast, only 3.0% of Entourage’s ratings are 1/10.
Yes, Wonder Woman wasn’t hated. That’s great. It doesn’t contradict the result that woman-led films encounter, in general, more hate than man-led films. As a next step, I would want to directly answer to the “what about Wonder Woman and Kill Bill” crowd, and dig into only the woman-led films. What variables do hated woman-led films share that non-hated ones don’t, and what variables do non-hated films share that hated ones don’t?
Horror and Auteur biases
Critically liked horror films and auteur movies seem to attract more hate than normal as well. From my experience, some horror fans just want to be scared or to have fun, and don’t care so much for a well-crafted narrative. That’s the best explanation I got for the horror hate. I find the results on auteur movies interesting as it’s the variable I created with the most personal discretion.
Auteur cinema has a specific definition in the film world (source), but I defined it a little differently to better leverage the data scraped from IMDb. Here, an Auteur movie is any film in which one of the writers directed the movie. That is, someone both (co-)wrote and (co-)directed the movie. Since this person has so much control over the film, one might expect Auteur movies to be more distinct, focused, and/or stylized. This can make bad movies more memorable, striking, or polarizing, explaining why they receive a particularly disproportionate amount of hate from IMDb users.
Limitations
This analysis makes some generalizations.
It generalizes the top 100 hated films, even though the range of hate can vary even within the top 100. It also misses some specific biases that are only visible for one or two films. For instance, The Promise (2016) is the critically mixed film with the second highest proportion of 1/10 ratings: 40.6%. This is not a Horror movie, nor is it woman-led. It’s a movie about the Armenian genocide that sparked a lot of political opposition from specific groups (genocide-deniers) that review-bombed the film. If you take a look at its entire rating distribution, it boasts a 48.7% of ratings that are 10/10 - making it one of the most beloved films on IMDb as well. Also in the top hated films, we see live-action remakes of animated classics, like Dragon-Ball: Evolution (2009) and The Last Airbender (2010). These examples are meant to illustrate the information we lose by looking at summary metrics and plots.
In this vein I encourage you to explore the top 100 most hated films (Table 2) and top 100 least hated (Table 3) per Metacritic Reception, and feel free to comment what you find:
Table 2: 100 Most Hated films on IMDb
Table 3: 100 Least Hated films on IMDb
| Rank | Unfavorable | Mixed | Favorable | |||
|---|---|---|---|---|---|---|
| Movie | Proportion of 1/10 IMDb Ratings | Movie | Proportion of 1/10 IMDb Ratings | Movie | Proportion of 1/10 IMDb Ratings | |
| 1 |  | 0.7 |  | 0.4 |  | 0.3 |
| 2 |  | 0.7 |  | 0.4 |  | 0.3 |
| 3 |  | 0.8 |  | 0.4 |  | 0.3 |
| 4 |  | 0.8 |  | 0.4 |  | 0.3 |
| 5 |  | 0.9 |  | 0.4 |  | 0.4 |
| 6 |  | 0.9 |  | 0.5 |  | 0.4 |
| 7 |  | 0.9 |  | 0.5 |  | 0.4 |
| 8 |  | 0.9 |  | 0.5 |  | 0.4 |
| 9 |  | 0.9 |  | 0.5 |  | 0.4 |
| 10 |  | 1 |  | 0.5 |  | 0.4 |