The Score: we’ve updated our weighted-rating calculations

Modern times call for an adjustment to our rating machine.
Modern times call for an adjustment to our rating machine.

Introducing a fairer approach to weighting, to better reflect the Letterboxd community’s global consensus for each film.

Today we’ve made the most significant change to our weighted-rating calculation since we launched in 2011. This means that, as of now, our official Top 250 looks a little different (with a new number one), as do several of our other official lists.

We’re doing this because, as our platform hip has grown (from a few tens of thousands in our first year to closing in on ten million today), we’ve become more susceptible to ratings distortion—some orchestrated, some not. Ratings distortion happens whenever the weighted average fails to capture the sentiment of our whole community for a film.

The Letterboxd weighted rating is a score that film fans across the globe have come to trust. It’s our responsibility to make sure it continues to be accurate and fit-for-purpose over a decade on, with some future-proofing built in. Read on for the whys, and more on what’s changed.

A snapshot of the top ten from our Top 250 Narrative Features list, maintained by Dave Vis.
A snapshot of the top ten from our Top 250 Narrative Features list, maintained by Dave Vis.

The previous approach

Up till now, we’ve applied a weighting to our computed average for each film, essentially to ensure that films can’t jump to the top of the charts with only a handful of high ratings; films with fewer member ratings were weighted more to our overall platform average than those with a higher count.

Despite this approach, our weighted averages could be swayed (up or down) by a concerted effort or campaign, or because a film was beloved in one region, or because superfans were attracted to it first, or due to any number of other scenarios. In cases of intentional manipulation, we’ve previously resorted to making manual adjustments to ratings eligibility; our changes today are designed to mitigate against all these scenarios.

The new approach

Today, we’ve introduced a better, fairer approach to weighting across the board, to more accurately reflect the Letterboxd community’s global consensus for each film.

Every member who rates a film is included, but we now do a better job of identifying unusual patterns of rating activity, and ing for them in our final score. As before, only a single rating per member is considered for each film; this is the last rating you made, if you’ve rated a film multiple times! 

Our specific approach remains private to protect its integrity, and is subject to change over time as we continue to adjust and improve it. We’ll announce major changes, like this one, should they be required in the future, so you know what’s happening.

Live footage of our engineering wing, running some calculations.
Live footage of our engineering wing, running some calculations.

The result

Any change to a long-standing system is bound to shake things up and, sure enough, this update has moved a few films around on several big lists that many of you track closely, including our official lists.

Excitingly, we’re christening a new number one of all time on the Top 250, as Masaki Kobayashi’s subversive samurai masterpiece Harakiri slices over Come and See and Spider-Man: Across the Spider-Verse to take the throne (both of the latter retain top-ten spots). Harakiri is the sixth film in our history to hold this coveted position.

Masaki Kobayashi’s 1962 film Harakiri slices its way to the number-one position in the Top 250.
Masaki Kobayashi’s 1962 film Harakiri slices its way to the number-one position in the Top 250.

Elsewhere, Letterboxd faves such as Cure, Secrets & Lies and Millennium Actress make their debut on the Top 250, while stone-cold classics Lawrence of Arabia, M and Ran rise up the ranks. The Top 250 Documentaries list, now topped by Stop Making Sense, especially benefits from a boost of populist and Oscar-winning titles such as Jiro Dreams of Sushi, The King of Kong: A Fistful of Quarters and Citizenfour.

This update also means increased overall tallies for legends highest-rated view, which tracks the highest-rated titles overall (of any format), was primarily dominated by music and TV-oriented titles; our improved method means there’s now more representation from narrative feature films.

The first ten films in our 2023 weekly top 50 (at time of publication).
The first ten films in our 2023 weekly top 50 (at time of publication).

What happens next

We expect films that debut in a single territory or at a film festival to take a little longer to settle into their long-term scores compared to how quickly they may have spiked in the past. You’ll still see important new independent films showing up in our main lists—take a look at the Top 50 Films of 2023, updated weekly, for an example of where some of this year’s best indies sit alongside major blockbuster releases.

Meanwhile, global day-and-date releases shouldn’t see as much of a change (aside from the normal settling we see as the ratings from a dedicated fan base are ed by those of a more general audience following opening weekend). Which reminds us: have you seen our new release-date feature? We’ve added release dates to film pages on the web, and they are coming to our apps very soon.

Further Reading

Tags

Share This Article