This Week In Game Shows Spotlight: Jeopardy! The IBM Challenge Recap

Ken Jennings "sucking up" to Watson

This past week marked two special events: Valentine’s Day and the Jeopardy! IBM Challenge. This three-day event took place from February 14 until February 16, 2011 at the T.J. Watson Research Center in Yorktown Heights, New York. This ultimate showdown pit IBM’s supercomputer Watson against two Jeopardy! super champs, $2.5 million winner Ken Jennings and $3.2 million winner Brad Rutter. The engineers at IBM decided to try this experiment because they wanted to try something bigger than what they have done before such as IBM’s “Deep Blue” defeating grandmaster chess champion Gary Kasparov in 1997. Because Jeopardy! has clues that often have subtitle meanings, semantics, and riddles within them, this would provide the creators of Watson with the ultimate challenge of programming him with the knowledge to respond to the clues according to how it's worded, something a regular computer may not pick up automatically.  In this week’s article, I am going to break down the highlights of each show.

The inner-workings of the 1.5 trillion byte
supercomputer Watson 

Watson was named after IBM founder Thomas J. Watson. He was built by a team of IBM scientists and engineers. Their goal was to build a supercomputer system that rivals a human’s ability to answer questions posed in natural language with speed, accuracy, and confidence. Watson has played more than 100 games of Jeopardy! against former contestants to prepare him for the pending battle. Watson may appear to be an unstoppable force, but like every electronic device that has ever been conceived, Watson does have some weak points. According to PBS producer Michael Bicks, Watson[‘s system that interfaced between Watson and the Jeopardy! computer] crashed a bunch of times. “It took over four hours to tape the show—most of the delays were due to crashes” (1) Bicks said. The winner of the three-day, two-game event will receive $1,000,000 cash. If Jennings or Rutter wins, they agreed to give half their winnings to the charity of their choice. If Watson wins, IBM agreed to donate all of their winnings to the charity of their choice.

And now, for the recap………

DAY 1

If Watson's confidence level does not reach the buzz
threshold, he will not buzz-in to respond to that clue

The show begins with Alex Trebek and announcer Johnny Gilbert giving a brief bio about Jennings, Rutter, and Watson. The six categories in the first round of the first game used were Literary Character APB, Beatles People, Olympic Oddities, Name the Decade, Final Frontiers, and Alternate Meanings. Since Rutter won the drawing backstage, he won the chance to choose the first category and clue. Even though Rutter buzzed-in with the first correct response, Watson quickly took control of the game, including his answering the Daily Double clue correctly for $1,000 on the third question. After about three minutes of gameplay, Watson was in first with $5,200, Rutter was in second with $1,000, and Jennings was in third with a surprisingly low $200. At this point in the game, it was obvious that the creators of Watson meant business and left no room for errors when creating him.

The first round game board in the first game

After the show returned from its second commercial break, Trebek discussed how Jennings’ and Rutter’s previous appearances on Jeopardy! changed their lives and how Watson started off on a rocky, almost unpromising start when playing some practice rounds of Jeopardy! After the brief interview with the contestants, the game resumed with Watson picking up where he left off. During the second half of the round, Jennings and Rutter had their chance to redeem themselves after a dismal performance in the first half of the round as Watson answered four clues incorrectly. Even though Watson lost a total of $3,000 in the first round, he was still able to tie Rutter for first place with $5,000 and Jennings was once again in the third place position with $2,000 at the end of round one. The end of this round concluded the first of three shows.

DAY 2

On the second show, the three contenders resumed their game by playing the Double Jeopardy round in the first game at the top of this game. But before the game continued, some of the IBM researchers discussed the other “grand challenges” they had accomplished, such as creating a supercomputer that can defeat any human grandmaster in chess (“Deep Blue”), providing the technology to land a man on the moon, and creating a computer system to map the human genome (“Blue Gene”).

In the second round, the categories were ‘Etude, Brute’, Hedgehogpodge, Don’t Worry About It, The Art of the Steal, Cambridge, and “Church” & “State”. Ken Jennings began the round because he was in third place in the first round.  Watson once again took control of the game during the first half of the second round answering 13 clues correctly, including both Daily Doubles for $6,435 and $1,246, and answering only clue incorrectly losing $1,600. Jennings only answered one clue right and one wrong while Rutter answered the only question he buzzed-in on incorrectly in this portion of the game. At the end of the first half of Double Jeopardy!, Watson stormed into the lead with $23,881, while Rutter was in second with $3,400 and Jennings once again in third place with an embarrassing $1,200. When the game resumed, Rutter answered his first clue correctly after Jennings selected the first clue since the last commercial break. In this portion of the game, there were no incorrect responses given. Watson answered 10 more clues correctly, adding another $12,800 to his total, while Jennings and Rutter both answered only two clues correctly. At the end of the second round, Watson maintained his first place spot with staggering $36,681; Rutter was in second with $5,400, and Jennings took third with $2,400. On a quick side note, this would be a similar scoreboard if Jennings was playing against to other contenders about six years ago, with Jennings being in the first place position of course. At this point, I think it’s safe to say who the winner would be; but, we still have one more round to deal with: Final Jeopardy!

All three contenders thinking about their "Final
Jeopardy!" response

The Final Jeopardy! category was U.S. Cities, and the clue was “It’s largest airport is named for a World War II hero; It’s the second largest, for a World War II battle.” Thirty seconds later, the contestants’ responses and wagers were unveiled. Since Jennings was in last place, we would see his answer first. Jennings responded with “What is Chicago? which turned out to be the correct response. His wager was $2,400, doubling his total to $4,800. Rutter answered with the same correct response as well. His correct answer added $5,000 to his total, bringing up him to $10,400 for the day. Watson, however, when on a different route and responded with “What is Toronto?????” This bizarre and incorrect response lost the supercomputer only $947, bringing his one-day total to $35,734.

Watson's infamous "What is Toronto?" response

Before I resume, I have to comment on Watson’s “Final Jeopardy!” response. David Ferrucci, the manager of the Watson project at IBM Research explained why Watson was so off with his response. During his training phase, Watson was taught that the category names are tricky and often misleading when it comes to figuring out what the correct response to the clues are in the category. Watson was taught how the categories work in Jeopardy!, understanding some of the subtleties in the clues in the game and not making simple assumptions based on the wording of the clue. The way the Final Jeopardy! clue was phrased provided an advantage for the human contestants and a slight disadvantage to Watson. However, if the words “U.S. City” were a part of the clue, Watson would have given U.S. cities much more thought as it searched for the answer. There are also cities named Toronto in the U.S. (as in the city in South Dakota, Iowa, Missouri, etc.) and the Toronto in Canada with a U.S. baseball team (The Toronto Blue Jays) that further confused Watson by possibly picking up those facts while analyzing his response. Watson also couldn’t find a clear-cut correlation between Chicago or Toronto’s airport and World War II. With Watson’s smart, meager wager of $947, the number of question marks he used in his response, and 30 percent confidence level about his response, it was clear that Watson was merely taking a guess at the clue.

Watson’s success in the first game made Rutter and Jennings seem like mere amateurs at the game. However, this game also proved that Jennings and Rutter aren’t terrible Jeopardy! contenders, but Watson was just faster at getting to the buzzer first after each clue had been read. It’s not over yet as there is one more game to play with $1,000,000 at stake this time around.        

DAY 3

On the final day of the two-game exhibition match, we re-enter this competition with Jennings and Rutter desperately needing to make some kind of comeback to keep themselves in this contest.

Jennings' and Rutter's former appearance on the
Jeopardy! Ultimate Tournament of Champions

This episode begins with Watson choosing the first clue of the second game. The categories in the first round were “EU, The European Union”, Actors Who Direct, Dialing For Dialects, Breaking News, One Buck or Less, and Also on Your Computer Keys. After the first half of the first round, Watson took first place once again with $4,200 by only a slight margin of $800. Jennings was in second this time with $3,800 and Rutter took third with $800. When the show came back from its first commercial break, Trebek discussed the prize money at stake in this game: $1,000,000 for first place, $300,000 for second, and $200,000 for third. Jennings agreed to give half his winnings to Village Reach, a Seattle-based non-profit organization; Rutter agreed to give half his winnings to his donor-advised fund at the Lancaster Community Foundation in Lancaster, Pennsylvania; and IBM agreed to donate all their winnings to World Vision and World Community Grid. In the second half of the first round, it seems like the momentum shifted from Watson to Jennings after Jennings doubled his total from $3,600 up to $7,200 on a Daily Double clue that he almost missed. Speaking of Jennings, he ended the first round with a decent lead over Watson having $8,600, Watson was in second with $4,800 and Rutter took third this time with $2,400. Jennings and Watson both answered 10 clues correctly in the first round, while Rutter answered only eight clues correctly.

Jennings' Daily Double clue in the first round

In “Double Jeopardy!”, the categories were Nonfiction, Legal “E”s, What to Wear?, U.S. Geographic Nicknames, Magical Mouse-tery Tour, and Familiar Sayings. Brad would begin this second round because he was in last place in the previous round. In this round, you will see Jennings giving Watson a run for his money as Jennings reverted back to his old ways, leading for most of this round. Also in this round, there was only one incorrect answer given and that was by Watson on the first Daily Double clue he picked, losing only $2,127 on that clue. Even though Jennings put up a better fight in this round, Watson still led the pack with $23,440. Jennings was in second place with $18,200 and Rutter was in third with $5,600. Only one round remains that separated the victor from that $1,000,000 grand prize: the Final Jeopardy! round. 

A more confident Final Jeopardy! response from
Watson

The Final Jeopardy! clue in this game was “19^th Century Novelists”. The clue was “William Wilkinson’s ‘An Account of The Principalities of Wallachia and Moldavia” inspired this author’s most famous novel.” Thirty seconds later, the contestants’ responses and wagers were locked in. Rutter, in third place, responded with “Who is Bram Stoker?”. This correct response doubled his total to $11,200, as he bid $5,600 on the clue. This brought his two-day total $21,600. Second place Jennings responded with “Who is Stoker? (I for one welcome our new computer overlords)”, adding a little humor to the already decided match. Jennings only wagered $1,000, giving him $19,200 for the day and $24,000 for his two-day total. This time around, Watson confidently gave the response “Who is Bram Stoker?”. He wagered a whopping $17,973 on this clue, bringing his total to $41,413 and his two-day total to $77,147. This made Watson the definitive champion of the Jeopardy! IBM Challenge. The $1,000,000 prize will be split between IBM’s charities, World Vision and World Community Grid, while Jennings took second place with $300,000, and Rutter took third with $200,000.    

Move over Ken & Brad! There's a new Jeopardy! 
super champ in the building!

This event will definitely go down in game show history as one of the most memorable moments to ever air on television. The engineers and scientists at IBM deserve much applause for creating a system such as Watson. The three-day match also brought in some incredible television viewer ratings. The first episode featuring the IBM Challenge aired on Monday drew an 8.7 household rating (about 14 million viewers), a 9.5 household rating on Tuesday’s show, and a 9.1 household rating on Wednesday’s show. Tuesday’s show marked the show’s highest rating since May 25, 2005, the same time Jennings and Rutter faced each other in the Ultimate Tournament of Champions (2). Wednesday’s show was the second-highest rated show on television that night, coming only second to FOX’s American Idol. But one more question remains as I close this article: What will Watson do next?


If you want to see Watson in action, click the links below to see all three days of the Jeopardy! IBM Challenge.
Day 1- Part 1   Part 2
Day 2- Part 1   Part 2
Day 3- Part 1   Part 2


(1)- "Despite Reports, Watson did not crash during Jeopardy taping" by Chris Matyszczyk- CNET News
(2) "IBM Watson: Jeopardy Champ, Ratings Winner" by Paige Albinaik- Broadcasting and Cable
"IBM Watson Puts Human Rivals In Jeopardy" by Paul McDougall- Information Week
"What is Toronto?"- IBM Webpage
**All screenshots have been taken from the actual episodes of Jeopardy! No ownership is implied.**