Art + Math + Science + English
  1. Mathematics
  2. Class 12
  3. Project 23
Mathematics Through Innovation and Problem Solving

23Spot the Scam: Building a Simple Message Classifier

Scam SMS and social media messages trick many people. Teams build a tiny "AI" that decides whether a message is a scam by counting suspicious words. Using a set of practice messages they write themselves (half normal, half scam-style), they calculate how often each word appears in each group, turn these into scores using probability, test the classifier on messages it has not seen, and discuss where it fails and why human judgement still matters.

Class and age
Class 12 · 17–18
Activity type
Group
Time needed
2–3 sessions of 60 minutes
Difficulty
Challenging
Estimated budget
Almost free

Mathematical concepts

  • Conditional probability
  • Bayes' idea: updating belief with evidence
  • Training and test data
  • Accuracy, false positives and false negatives
  • Bias and limits of AI

Learning objectives

  • Estimate conditional probabilities from counts.
  • Build a simple word-score classifier.
  • Test a model fairly on unseen data.
  • Explain the strengths and weaknesses of automatic filters.

Real-life connection

Email spam filters, fraud detection in mobile banking and content filters on social media all started from ideas like this one. Knowing how they work helps people protect themselves and their families from scams.

Materials required

  • Index cards for writing practice messages
  • A spreadsheet
  • A list of common scam warning signs from a trusted source (chosen with the teacher)

Step-by-step instructions

  1. As a class, write 40 short practice messages: 20 normal (from friends, school, family) and 20 scam-style (fake prizes, urgent requests for PINs or OTPs). Never copy real messages that contain personal details.
  2. Split them: 30 for training and 10 kept aside for testing.
  3. In the training set, count how many scam and how many normal messages contain each key word (for example "prize", "urgent", "PIN", "click", "tomorrow", "class").
  4. For each word estimate P(word | scam) and P(word | normal). Add 1 to every count so no probability is zero.
  5. Give each word a score: the ratio P(word | scam) ÷ P(word | normal). For a message, multiply the scores of its words (or add their logarithms).
  6. Choose a cut-off: above it, label the message a scam. Classify the 10 test messages.
  7. Fill a confusion table and calculate accuracy, false positives (normal messages flagged) and false negatives (scams missed).
  8. Try to write a scam that fools your classifier. Then write a short guide for families on scam warning signs that no filter can replace.

Mathematical explanation

If a word appears in many scam messages but few normal ones, seeing it is evidence for "scam", and its score ratio is above 1; common friendly words have ratios below 1. Multiplying the ratios combines the evidence from all the words, which is the idea behind the naive Bayes classifier (it is "naive" because it treats words as independent). Adding 1 to every count stops a single unseen word from making a probability zero. Testing on messages the model never saw is essential, because a model can memorise its training data. Moving the cut-off trades false positives against false negatives. Scammers adapt their wording, so filters must keep learning, and people should still check any message that asks for a PIN, OTP or money.

Expected project output

A training table of word counts and scores, a working spreadsheet classifier, a confusion table on test messages, and a family guide to spotting scams.

Skills developed

  • Conditional probability
  • Building and testing a model
  • Digital safety
  • Critical thinking about AI

Assessment criteria

  • Word probabilities are estimated correctly, with the add-one rule.
  • The classifier is built and applied consistently.
  • Testing uses only unseen messages and results are reported honestly.
  • The family guide is clear, practical and accurate.

Extension challenge

Write the classifier in Python, or add word pairs (for example "send PIN") as features. Does accuracy improve on new test messages?

Safety and inclusion

Use only messages the class writes. Never click links, call numbers or reply to real suspicious messages, and never share real PINs, OTPs or phone numbers.