Blankety Blank AI Prediction Machine Activity

A monkey holding a banana. — Image by Dmitry Abramov from Pixabay

This activity explores how chatbots like ChatGPT work. ChatBots, based on what are called Large Language Models, are able to generate text in response to prompts, including writing essays and having conversations. They can seem to be incredibly intelligent as a result but all they are ultimately doing is predicting what text comes next. This is a simple unplugged activity to illustrate that and how it can generate wrong “hallucinated” facts.

It is also a way to review particular topics where a lot of students have misconception and could be used as a tallying exercise.

Learn about what?

The basics of Artificial Intelligence ChatBots (Large Language Models) work.
How they are predicting text that comes next.
Why they get things wrong.
Bar charts

Ask what the class know about AIs and Chatbots. If you have suitable access then demonstrate using a chatbot being asked questions live. Explain how Chatbots appear to be intelligent but are actually just making predictions about what comes next based on the documents they were trained on. This is often social media or the web, where the information found can be anything that the author wrote. It may or may not be correct. The AI gathers all this information (in advance) and works out how often words come next in common sentences. When answering questions it just uses those predictions picking one of the most common words or phrases to go next.

Demonstrate this with a class activity based on the Quiz show Blankety Blank.

Blankety-Blank

You present the class with a series of unfinished sentences in turn. For each the students must decide the word or phrase they would put next to fill the blank.

For example, given the start of the sentence:

Birds _________

Answers might be “fly”, “lay eggs”, “have feathers”. “sing”, …

Each student writes their idea for the word to fill the blank on a post-it note. Explain they are playing the parts of source documents for the chatbot, with information, but as there are lots of them there are lots of possible answers.

Collect in all the answers and stick them on the board, grouping them in columns of exactly the same word/phrase to make a bar chart. From the bar chart you can see which answers are most common. Point out there are many different phrases that could come next, taking the text in completely different directions from that point. A chatbot has lots of possibilities.

Pick out the top two from the barchart, noting they have greatest frequency so are good candidates for what goes next for the chatbot. Now, however, toss a coin with Heads for one choice and tails for the other. Complete the sentence on the board with that choice and say that is what the class chatbot has chosen.

Point out there were lots of possible answers but one of the most popular is chosen, but at random. That is why a chatbot comes up with different answers each time it is asked as it works by the statistics of which is most likely but still uses some randomness (not just between the most popular two but from hundreds or thousands of popular words to come next).

Discuss whether all the choices that were candidates were good answers or even actually true. For example, given the sentence to complete:

Monkeys eat _________

The most popular answer might be bananas, but actually monkeys eat all sorts of fruit and nuts, seeds, insects, animals, lizards, eggs, and more, so bananas is a bit misleading. Giving a list would have been better.

If the answer given to the following is “monkeys”, which many people beleive, then actually that is not a true sentence generated.

Lemurs are monkeys. Baboons are monkeys. Orangutans are _________

Explain that this is why chatbots get some things wrong so hallucinate. They just base answers on what is popular in the documents they have been trained on, so if lots of people have written that Orangutans are monkeys on webpages or in social media, then that could be what the chatbot generates as the next word even though it is not true.

You might also note that if the documents ised fro training contain horrible (racist, homophobic or similar) sentences then the chatbot may choose similar words and also say horrible things.

Here are some other possible sentences with blanks to use:

Zebras are _________

Giraffes have long  _________

All birds can ________

The fastest animal on Earth is the  ________

Mammals do not  ________

Birds eat _______

When you visit the pond, you should feed the ducks ________

Explain that a chatbot is making predictions like this for every word, not just the end of sentences, but based on the words in the prompt it was given.

Have the class summarise what they know now about how a Chatbot works. To finish, summarise the main points, that:

a Chatbot is just predicting what is the most likely words, phrases, sentences and paragraphs to come next
It makes the prediction based on how likely that word is to come next based on the documents it has trained on (essentially read). Each student was acting like one of those documents each with different possible words,
It chooses the next word from the popular ones, but may not pick the most popular, as at this point it chooses at random to some extent.
Depending on the documents it was trained on some answers might be right, some wrong, some right but perhaps not the most accurate or illuminating answer.

Note that, alternatively you can do the exercise on some topic that you need to cover anyway with particular misunderstandings that you want to discuss anyway.