The Netflix Prize: How Recommendations Learned Your Taste
Two teams tied after three years, and a million dollars went to whoever pressed send twenty minutes sooner. Build the machine they raced, then try to beat it.
You are a row. Not a person — a row. You are one horizontal strip of a spreadsheet so wide it does not fit on any screen ever built: 17,770 columns across, one for every film Netflix rented by post, and 480,189 rows down, one for every anonymous customer whose ratings were dumped onto the internet on 2 October 2006.
Your row has about 209 numbers in it. Every other cell — 17,561 of them — is blank. Somewhere out there, forty thousand teams of strangers are staring at the blanks in your row and trying to guess what you would have said.
Netflix offered a million dollars to anyone who could guess ten per cent better than Netflix could. It took three years. It ended in a photo finish. And the thing the winners built — a little machine that learns to describe you and every film in the same handful of numbers — is now quietly running underneath almost everything you are recommended, everywhere.
This page contains a working one. You will rate some films, watch it train, drag your own taste around a map, and then find out why the winning code was never switched on.
Act One · the blanks
Multiply it out and the grid has 8,532,958,530 cells. Netflix released 100,480,507 ratings. That is 1.18% full. The entire competition was an argument about the other 98.82%.
Below is a real slice of a matrix at exactly that density — 220 users across, 120 films down. Drag or move your pointer over it to inspect a cell. The bright specks are ratings that exist. Everything charcoal is a question nobody has answered.
If you printed the real grid one cell per square millimetre it would cover about 8.5 square kilometres. Roughly a hundred square metres of it would have anything written on it. Every recommender that has ever existed is a proposal for filling in the rest.
Netflix's own predictor was called Cinematch. Scored on a hidden set of ratings, it was wrong by an average of 0.9514 stars — that is the root-mean-square error, or RMSE, and lower is better. Guessing each film's average rating for everybody scores 1.0540. Guessing the single global average, 3.6033 stars, for absolutely everything scores about 1.13. So Cinematch, the state of the art at a company built on recommending things, was under ten per cent better than "just use the film's average."
Beat it by another ten per cent — get down to 0.8572 — and a million dollars was yours.
Careful with those numbers: 0.9514 is Cinematch on the public quiz half, 0.9525 on the secret test half. Both get quoted as "the" baseline. They are two different halves of the same envelope.
Act Two · the two-number trick
On Monday 11 December 2006, ten weeks into a three-year contest with a million dollars on the table, a programmer sitting in third place published exactly how he was doing it. His name on the site was Simon Funk. His real name is Brandyn Webb. He opened like this:
He never won a cent. Almost every leading solution for the next three years descended from that post.
His idea is the whole lesson, and it is small enough to hold in your hand. Stop trying to describe films with genres. Give every film a short list of numbers — say two of them — meaning nothing in particular. Give every person a list of the same length. To predict what a person will think of a film, multiply the lists together and add up.
μ is the average rating of everything. bu is how generous you are. bi is how loved the film is. qi and pu are the two lists of numbers — the film's and yours — and the dot means "multiply matching entries and add them up." Nobody chooses what the numbers mean. They fall out of the arithmetic.
And here is the part that feels like a magic trick. Nobody tells the machine what the numbers should be. It starts with nonsense, makes a prediction, sees how wrong it was, and nudges both lists a little in the direction that would have been less wrong. That is it. That is the entire learning rule Funk published:
γ (gamma) is how big a nudge. λ (lambda) is a leash that keeps the numbers from running off to infinity by memorising instead of learning. On the Netflix data the winners used γ = 0.005 and λ = 0.02. You will get to move both in a moment.
Act Three · your turn
Here are twenty films. They do not exist — I invented them, and drew every poster in code — but sixty invented strangers have already rated them, and their opinions have a real hidden structure buried in them, exactly the way a real ratings matrix does. Rate at least five. Be honest; there is no right answer and nothing leaves this page.
Press start. Every "epoch" is one pass over every rating in the matrix, applying Funk's two lines to each one. The map on the left is the films arranging themselves in a space nobody designed. The chart on the right is the error falling — and, if you are careless, rising again.
Each dot is a film, placed at its first two learned numbers. The red diamond is you — and you can drag it. Drag it and the predictions below re-sort instantly. That is not a simulation of a recommender; that is literally what the arithmetic does.
RMSE in stars. Lower is better.
Predict first: take the leash off (λ = 0, f = 6) and train for 100 epochs. What will the error on hidden ratings do?
Challenge: tune f, γ and λ until the error on hidden ratings drops below 0.77 stars.
Not trained yet. The numbers inside every film and every viewer are currently random noise.
These are predictions for the films you did not rate, in the machine's order of confidence. Drag the red diamond on the map and watch the list rearrange — you are moving your own taste vector by hand.
1. Take the leash off. Set λ to 0 and f to 6, then train for a hundred epochs. The "seen" error dives beautifully towards zero and the "hidden" error turns around and climbs. The machine has stopped learning about taste and started memorising individual ratings. That is overfitting, and you can watch the exact epoch it begins.
2. Nudge too hard. Leave λ at 0 and f at 6, and now push γ past 0.06. The hidden error goes worse than simply guessing each film's average. Learning is a walk downhill; take strides that are too long and you bounce off the opposite wall of the valley instead of walking down it.
3. Give it one number. Set f to 1. Now every film sits on a single line and every person is a single number, so the only thing the machine can say is "you like the kind of film that people like you like — more, or less." It gets surprisingly far. It cannot ever separate two people who agree about spectacle and disagree about sweetness. The second number buys you that.
When Yehuda Koren, Robert Bell and Chris Volinsky published how they did it, they printed a picture of the first two learned numbers from the real Netflix data, with a scattering of real films placed on it. The axes on that figure have no names. They are called “Factor vector 1” and “Factor vector 2”, because that is genuinely all they are.
Then the humans looked at the picture and gave it names anyway. Here is their own caption:
Below is a drawing of that figure, rebuilt in code from the film names printed in the paper. Hover a title. Nothing in the algorithm knows what “fraternity humor” means — it only knew who watched what. The meaning is something we put on afterwards, and that habit is worth being suspicious of every time you meet it.
A coded redrawing of Figure 3 of Koren, Bell & Volinsky (2009). Positions are approximate; the film names are the paper's.
Two of the films the paper puts near the “quirky independent” corner are Lost in Translation and Being John Malkovich; near the opposite corner sit Road Trip and The Fast and the Furious. The machine had never heard of either category.
In November 2008 The New York Times Magazine followed a competitor named Len Bertoni, and found him stuck on a single film: Napoleon Dynamite. His predictions were typically off by about eight-tenths of a star. On that one film he was off by 1.2.
The problem was not that the film was obscure. It had plenty of ratings. The problem was that people gave it one star or five stars and almost nothing in between. Two viewers whose taste vectors were nearly identical would split on it. There is no set of numbers that can predict a coin flip.
Your twenty invented films contain one of these on purpose. It is called Everything Is Fine. Look for it on the map — the machine will place it somewhere bland and confident and be wrong about it forever. Some of the error in every model is not ignorance. It is genuine unpredictability, and knowing which is which is most of the job.
Game · you against the machine
Ten rounds. Each round shows what one invented viewer rated, and one film whose rating is hidden. Pick their stars before the bar runs out.
A matrix-factorization model trained on everyone else guesses too. An exact guess scores 3, within one star scores 1 and keeps your streak alive. At the end, compare your error with the machine's.
Press start. The clock only runs while a question is open, and you can play as many times as you like.
This is the whole competition on one ruler. Everything is RMSE on the public quiz half, in stars, and the entire prize lives inside a gap of 0.0947 stars — less than one tenth of one star on a five-star scale.
Netflix Prize quiz-set RMSE. The dashed line is the qualifying threshold, 0.8563.
Read the top two dots and then read the bottom two. Three years, forty thousand teams and a million dollars moved the needle from “wrong by 0.95 stars” to “wrong by 0.86 stars.” Whether that is a triumph or an anticlimax is the argument the rest of this page is about.
Act Four · the last forty-two minutes
A month earlier, on 26 June, a merged team called BellKor's Pragmatic Chaos — three teams from Austria, Canada, the United States and Israel who had never all been in the same room — finally crossed ten per cent. That triggered the rule everyone had been dreading: last call. Every other team on Earth now had exactly thirty days.
On 25 July, with one day left, a team that had not existed the week before appeared at the top of the leaderboard. The Ensemble was twenty-three separate teams who had fused themselves together for the endgame. Press play and watch the last forty-two minutes.
You will often hear that Netflix threw the winning algorithm in a drawer and never used it. Here is what Netflix's own engineers actually wrote, three years later:
That sentence is about the final blend — a stack built out of 454 separate predictors. Nobody was ever going to run that in a web page. But the same post says, in the same breath, that two things from the competition did go into production and stayed there: matrix factorization — the machine you just trained — and a second model called an RBM. Alone they scored 0.8914 and 0.8990; blended, 0.88.
So the honest version is stranger and better than the myth. Netflix kept the idea and binned the trophy. And then, in 2017, it did something even more startling: it deleted the five-star rating from its own product and replaced it with a thumb — because it turned out people gave documentaries five stars and then went and watched something silly.
The exact quantity that a million dollars and three years of the world's cleverest statisticians were spent minimising was retired eight years after it was won.
Act Five · the fingerprint
Netflix stripped the names off. Every row was a number. Within about three weeks of the data going up, two researchers at the University of Texas — Arvind Narayanan and Vitaly Shmatikov — had posted a paper explaining why that was not enough. Their finished version appeared at the IEEE Symposium on Security and Privacy in 2008.
Their argument is easy to feel and hard to escape. Nobody else has watched exactly what you have watched. Names are not the identifying part. The list is.
Try it. Below are the same twenty invented films, with how many of twenty thousand invented strangers have seen each. Click the ones you would have watched. Watch the crowd disappear.
Guess first: how many films will it take to narrow 20,000 people down to one?
Challenge: single out one person using 3 films or fewer. Hint: rare films cut deepest, and knowing roughly when someone watched cuts deeper still.
Twenty thousand people. Any of them could be you.
They took the anonymous Netflix rows and lined them up against ratings people had posted publicly, under their own names, on IMDb. Same films, roughly the same weeks, different websites. Two rows that agree that closely are the same person.
Be precise about the size of it, because this story gets inflated constantly. 99% is a statement about the shape of the data — how unique the rows are — not a count of people who were unmasked. Their live IMDb experiment used a sample of about fifty IMDb users, and identified two of them, with match scores 28 and 15 standard deviations above chance. Two people. That is quite enough: the point of a proof of concept is that anybody can now do it at scale.
In December 2009 a closeted lesbian mother, filing as Jane Doe, sued Netflix under a 1988 law originally passed because a newspaper printed a Supreme Court nominee's video-rental records. Netflix had already announced a bigger sequel competition with richer data. On 12 March 2010, after an understanding with the Federal Trade Commission and a settlement with the plaintiffs, Netflix cancelled it.
The most generous act in the history of machine-learning research — a company giving away a hundred million real observations so that strangers could improve on it — is also the reason nobody has done it since.
Every part of this story has a tidier version in circulation. Here is where the tidy version and the paperwork disagree.
Eight questions. Wrong answers are more fun than right ones.
You need: five or more people, a sheet of paper, a pen. Time: twenty minutes.
Rule a grid: people down the side, six things across the top — six songs, six foods, six films, six board games, anything you can all have an opinion about. Everyone fills in only four of their six boxes with a score out of five, and leaves two deliberately blank. Fold the paper so nobody has seen anyone else's row. Now unfold it. You have just built a sparse ratings matrix.
Round one — the baseline. For each blank, predict it as: the average of everything on the sheet, plus (that person's average − the sheet average), plus (that column's average − the sheet average). Write your predictions in pencil. Now reveal the true answers and, for each one, square the error, average the squares, and take the square root. That number is your RMSE, and it is exactly the number a million dollars was paid for.
Round two — find a factor. Sort the six columns by hand into an order that feels like a spectrum: sweetest to strangest, calmest to loudest, whatever the group agrees on. Give each column a number from −2 to +2 along that spectrum. Then give each person a number from −2 to +2 for how much they like that end. Multiply the two numbers, divide by four, and add that to your baseline prediction. Score it again. If your RMSE went down, congratulations — you have done, entirely by hand, the thing that took three years and forty thousand teams.
Boss level. Before you reveal anything, have one person secretly write down which four boxes each person filled in — not the scores, just which boxes. Then see whether they can identify who is who from that alone. They usually can. That is the whole of Act Five in ninety seconds.
The papers, posts and articles the facts on this page come from.
Built for the world_learning Daily Learning Experience, 21 August 2026. Every image on this page — the room, the posters, the maps, the charts — is drawn from scratch in code; there are no photographs. The twenty films and sixty viewers are invented, and flagged as such wherever they appear; every date, name, quotation and RMSE figure is real and sourced below. Nothing you type or click is stored, sent or remembered after you close the tab.