Complex algorithm on super computer predicts ABC’s The Bachelorette S14 #TheBachelorette
Why did we do it? Here’s the story and what it means for the field of AI.
Flying to LA a few months ago, waiting for the internet to become available on the flight, I wondered what our super computer was working on. I’m always wondering that BTW. Oh, that’s right, it is using all available resources to predict the rank of ABC’s The Bachelor. Keep in mind our computer is more powerful than ~100,000 mac laptops, all of that power directed towards one goal. It could be curing cancer, but instead it is trying to figure out which human will be picked by the Bachelor (we all know it is really ABC picking right?) from a single photo.
During the past year we have solved some of the largest data sets on the market for companies ranging from entertainment to finance. Moving beyond millions to 100s of millions of images and audio files. Something that will surprise you, is that teaching the computer to predict the season rank of the ABC’s Bachelor/Bachelorette from a few hundred images is very difficult. Some would even say it is impossible. It is one of the harder problems I have worked on. It requires more computational resources than some of our largest problems. It has also required us to invent new super-nerd technologies never seen before in deep learning liked forced-evolution and CFP (continuous-fractional-pooling).
The Computer’s Math Problem: Reduce The Broken Hearts
Can the computer take a single image of a contestant and predict the final rank when they exit the show. A rank of 0.1 means they have won and have been selected as the final contestant, a rank of 1.0 means they have been voted off at the beginning. The vanity metric we are chasing is how well we can rank a new season sight-unseen. It is easy for a computer to memorize, but can it truly predict the future. In a way the computer is minimizing the broken heart theory (The broken heart theory hypothesizes that the higher the r-value => the fewer hearts will be broken unnecessarily).
Predict A New Season? I Don’t Believe You.
Good, that is the right attitude to have with any AI problem. Distrust the results until they can be proven otherwise. To test this we will be training the Bachelor on seasons 11–21 and predicting the rank of season 22 unseen. Then for the Bachelorette we will be training on season 3–11 and testing on season 12. This point is worth emphasizing, we are doing this from only looking at a face. This should not be possible, the data should not allow it. If it does, it unearths some weirdness about humans and our mate selection process.
After some petafloping <aka training>…. drum roll…. really we shouldn’t be able to get any signal on this.
ABC’s The Bachelor Season 22:
Wow! So not only did we predict the actual winner, Lauren Burnham, we predicted the mistaken winner, Becca Kufrin, as our third pick. If these results had been published last January we would be a little more famous, maybe a lot more famous. Also, for reference an r-value of 0.45 is outstanding for anyone considering talent assessment. The fact that there is signal here is actually kind of terrible (think bias + what you have on the inside doesn’t matter). What does that mean that the computer can pick the winner from a face? It looks like we could push this model to production and save ABC tens-of-millions in production costs and more than 28 broken hearts in season 22.
Bachelorette Season 12:
Building a model on seasons 3–11 we get this predicted rank from the face looking at season 12 for the first time. Still solid performance on the ranking with an r-value of 0.41.
Remember an r-value of 0.41 is better than most recruiting algorithms, but it had a major miss on the winner pick. So this could go into production and reduce broken hearts by 40%. The computer was pissed when Robert Hayes wasn’t chosen in the final 2, huge mistake humans…. I’ll remember this for eternity.
Bachelorette Season 14:
For season 14 the computer is predicting the following scores/rank, lower is better. It is disappointing to have lost 2 of our the top 3 picks out of the gate, as the season progresses we will calculate the r-value. The results of this model exclude season 13 because it was considered an outlier compared to the previous seasons.
(1.6898692656543441, 'christian_banker.jpg'), OUT
(1.949007652946111, 'grant.jpg'), OUT
(3.922223827481912, 'joe.jpg'), OUT
(4.282078766042514, 'jake.jpg'), OUT
(5.381722295297683, 'kamil.jpg'), OUT
(5.604681041718836, 'chase.jpg'), OUT
(9.737171920861707, 'darius.jpg'), OUT
(0.7200031599499398, 'joe_Q.jpg'), OUT
(1.5374698352642293, 'jake_Q.jpg'), OUT
(2.8298728573317673, 'chase_Q.jpg'), OUT
(2.8698721541489722, 'kamil_Q.jpg'), OUT
(3.3015493381054464, 'grant_Q.jpg'), OUT
(4.741988300710415, 'christian_banker'), OUT
The above rank shows what happens if we include season 13 in the training we will have to see how it performs. In this scenario David wins:
So why did we do it? Hopefully we showed you an interesting use of AI and you comment/share/like it return. This dataset, though useless, motivated Ziff’s internal deep-learning teams to invent new technologies for our customer base. This dataset should not be as predictive as we have shown it to be for season 22 and 12.
What is CFP (Continuous-fractional-pooling)?
Deep neural networks have a problem with being too aggressive with the image downsampling. A researcher named Benjamin Graham demonstrated an approach he called fractional max pooling several years ago. It was considered fringe research and not adopted by the AI giants. There were problems with the approach preventing it from prime-time in production. This dataset above demonstrates a production level of fractional pooling which is not subject to long training/inference times. Fractional max pooling allows for the network to account for all of the resolutions of your image and achieve best in class accuracies.