← All Problems

8. Flag Likely Spam/Automated Emails

Confirmed Medium Rule-Based Classification
Grounding: Confirmed: For the email channel, Fin's dedicated component includes spam/automated-email detection so it doesn't try to answer non-customer messages. (Source: intercom.com/blog/fin-over-email-how-we-built/)

Problem

Not every email that lands in Fin's inbox is from a real customer with a real question. Automated bounce notices, out-of-office auto-replies, and marketing blasts also show up in the same channel, and Fin shouldn't try to generate a helpful answer to any of them. Given an email's subject and body, decide whether it looks like an automated/spam message rather than a genuine customer question, using a fixed deterministic rule set standing in for a trained classifier.

Source: src/8_is_likely_spam_email.py

def is_likely_spam(subject: str, body: str) -> bool

>>> is_likely_spam("Out of Office: John is away", "I will respond when I return.")
True

>>> is_likely_spam("Question", "Please help me now")
True

>>> is_likely_spam("Order Issue", "Hi, I ordered a laptop last week but it still hasn't arrived. Can you help me track it?")
False

Step-by-Step Approach

  1. Pick the strongest, cheapest signal first: a fixed set of subject-line keywords that almost never appear in a genuine customer subject — "out of office", "automatic reply", "undeliverable", "unsubscribe", "no-reply".
  2. Lower-case the subject once and check whether any keyword appears as a substring — subjects vary in exact wording ("Out of Office: John is away"), so substring match beats exact match.
  3. If any keyword hits, immediately classify as spam — this is a high-confidence rule, no need to look at the body.
  4. If the subject is clean, fall back to a weaker heuristic on the body: a genuine support question is usually either a real question (contains "?") or has enough words to be a substantive message.
  5. Flag as spam only when the body has no "?" AND is short (fewer than 5 words) — both conditions together, since either alone is too aggressive (a long questionless statement, or a short question, can both be legitimate).
  6. Otherwise, classify as not spam.

The key insight is layering rules from highest-confidence to lowest: a strong subject-keyword match short-circuits immediately, while the weaker body heuristic only fires when both signals (no question mark, too few words) agree, minimizing false positives on real customer messages.

Reference solution

SPAM_SUBJECT_KEYWORDS = {
    "out of office",
    "automatic reply",
    "undeliverable",
    "unsubscribe",
    "no-reply",
}


def is_likely_spam(subject: str, body: str) -> bool:
    # Two O(1)-ish checks: subject keyword substring match, or a short questionless body.
    # normalize case once before comparing
    subject_lower = subject.lower()
    # substring, not startswith
    if any(keyword in subject_lower for keyword in SPAM_SUBJECT_KEYWORDS):
        return True
    # split() collapses whitespace runs
    if "?" not in body and len(body.split()) < 5:
        return True
    return False

Key Functions & Tricks

  • {...} set literal (SPAM_SUBJECT_KEYWORDS) — O(1) average membership checks per keyword
  • str.lower() — normalizes subject once before any keyword comparison
  • any(keyword in subject_lower for keyword in ...) — substring match, short-circuits on first hit
  • "?" not in body — cheap single-character containment proxy for "is this a question"
  • body.split() — no-arg split collapses whitespace runs, used for approximate word count
  • Short-circuit early-return layering — checks ordered strongest-to-weakest signal first

How to Recognize This Pattern

Reach for a layered, deterministic rule-set classifier whenever a problem asks for a binary decision (spam/not-spam, valid/invalid, urgent/normal) based on a handful of independent heuristic signals rather than a full ML model — the tell is language like "fixed rule set" or "deterministic" standing in for a classifier. Order checks from strongest/cheapest to weakest, short-circuiting on high-confidence signals. A common variation is scoring instead of short-circuiting: assign each rule a weight and sum, then threshold, when signals should combine rather than override. A common pitfall is using a single weak signal alone (e.g. "body is short" by itself) as a hard rule — it needs to be combined with a second corroborating signal (no "?") to avoid misclassifying legitimate short questions as spam.