Data in Context

Concept Lesson

Imagine you are a data analyst at a Nigerian bank that processes ₦4.2 billion in mobile money transactions every day. Your team lead drops a CSV file on your desk and says, "Take a look at this before we build anything." You open the file and see 10,000 rows with four columns: amount in Naira, timestamp, merchant_category, and is_fraud coded as 0 or 1. No Python scripts, no Jupyter notebooks — just your pen, a calculator, and the reasoning skills you have built over the last two weeks. This is how every real ML project should start: with a human reading the data before a machine ever touches it.

Your first instinct should be to count. How many rows are there? The file says 10,000 — that is your dataset size, and it matters because small datasets lead to unreliable patterns. Next, you split the data by the is_fraud column. Suppose you find 9,400 legitimate transactions and 600 fraudulent ones. That is a 600 / 9,400 = 6.38% fraud rate, or a ratio of roughly 1 fraud case for every 15.7 legitimate transactions. This class imbalance is critical. If you ignored it and built a model that always predicted "not fraud," the model would appear to be 94% accurate — an impressive number that completely fails at its job. Your team lead would see a high accuracy score, approve deployment, and the bank would lose money every single day because the model never catches the fraud it was designed to detect. Understanding class balance before modeling is not optional; it is the difference between a useful system and an expensive paperweight.

Now look at the amount column. Calculate the overall mean and median transaction amount. Suppose the mean is ₦18,500 and the median is ₦4,200. That gap is a red flag — it tells you the distribution is right-skewed, pulled upward by a few very large transactions. Now split by fraud status. Perhaps legitimate transactions have a mean of ₦12,000 and a median of ₦3,800, while fraudulent transactions have a mean of ₦95,000 and a median of ₦72,000. The fraud amounts are dramatically higher, but the median is the more honest measure here because it is not distorted by a single ₦2,000,000 transfer. If you set a threshold for suspicious transactions based on the mean, you would flag every transaction above ₦95,000 — but that misses the cluster of fraud cases in the ₦50,000 to ₦80,000 range. The median-based approach would catch more real fraud. Choosing mean versus median is not a technicality; it determines whether your alert system actually works or just looks good on a report.

Finally, examine time. Sort the timestamps into weekly buckets and count the fraud cases per week. Suppose Week 1 had 80 fraud cases, Week 2 had 95, Week 3 had 110, Week 4 had 130. That is a consistent upward trend — roughly 15–20 additional fraud cases each week. If you calculate the rate of change from Week 1 to Week 4, fraud increased by (130 − 80) ⁄ 80 = 62.5% over four weeks. This tells you the problem is getting worse, not better, and any model you build must be retrained regularly or it will drift (meaning the patterns in the real world change over time — for example, fraud tactics evolve — so a model trained on last month's data starts making wrong predictions on this month's data) out of date within a month. A model trained on Week 1 data alone would miss the evolving fraud patterns appearing in later weeks. All of this analysis — ratios, percentages, central tendency, spread, and rate of change — comes directly from the first two weeks of this course. You are now applying those tools to make a real business decision.

Concept Lesson

Imagine you are designing a spam filter for an email service used by 2 million Nigerians. Every day, about 800,000 emails flow through the system, and roughly 120,000 of those are spam. That means the probability that a randomly selected email is spam is 120,000 / 800,000 = 0.15, or 15%. A probability is simply a number between 0 and 1 that describes how likely something is. A value of 0 means the event is impossible; 1 means it is certain; 0.5 means it is a coin flip. Every ML classifier you will ever use outputs probabilities under the hood, even if the final prediction you see is just a class label. When your spam filter says "spam," it first computed something like P(spam | email content) = 0.87 and then decided that 87% confidence was high enough to flag the message. Understanding what those numbers mean is the foundation of building systems that make good decisions under uncertainty.

Now consider two things happening together — this is called joint probability. Suppose there is a 30% chance of rain in Lagos tomorrow, and a 50% chance you forget your umbrella on the bus. If these two events are independent, meaning rain does not affect whether you forget your umbrella, then the probability of both happening is 0.3 × 0.5 = 0.15, or 15%. You multiply the individual probabilities because you need both conditions to be true simultaneously. This seems simple, but it matters enormously in ML. When a fraud detector considers multiple suspicious features at once — a large transaction amount AND an unusual merchant AND a new device — the joint probability of all three happening by coincidence is much smaller than any single feature alone. Models learn to exploit these joint probabilities. The danger is when features are correlated: if "unusual merchant" and "new device" tend to appear together, treating them as independent inflates your confidence. Ignoring correlation between features is one of the most common ways ML models produce badly calibrated predictions. But in ML, features are often NOT independent — if a customer's spending increases, their transaction count usually increases too. When events are correlated, you cannot simply multiply their probabilities.

Conditional probability is the engine that makes ML work. It asks a targeted question: given that I have already observed event X, what is the probability of outcome Y? This is written P(Y | X). For your spam filter, P(spam | email contains "winner") might be 0.72, while P(spam | email contains "meeting") might be only 0.03. The overall probability of spam is 15%, but the word inside the email changes that probability dramatically. Your filter works by computing conditional probabilities for every word and feature in the email, then combining them to produce a final score. This is exactly how a fraud detection system works too: P(fraud | amount > ₦100,000) is much higher than P(fraud | amount = ₦2,000). Conditional probability is the bridge between raw data and meaningful predictions.

To see why conditional probability can be deeply counterintuitive, consider this medical screening scenario. A hospital in Abuja screens 10,000 patients for a disease that affects 1% of the population. The test is 95% accurate: if you are sick, it says positive 95% of the time; if you are healthy, it says negative 95% of the time. Sensitivity (also called recall) is the chance the test catches you if you are actually sick — in this case, 95%. Specificity is the chance the test correctly clears you if you are healthy — also 95% here. Out of 10,000 people, 100 have the disease and 9,900 do not. Of the 100 sick people, 95 test positive (true positives). Of the 9,900 healthy people, 5% test positive by accident — that is 495 false positives. So out of 590 total positive results, only 95 are real. The probability of actually having the disease given a positive test is 95 / 590 = 16.1%. Your gut says 95%, but the math says 16%. This gap between intuition and calculation is exactly why ML practitioners must think in probabilities rather than in gut feelings. The consequence of getting this wrong in a real hospital is that hundreds of healthy patients receive unnecessary treatment, wasting resources and causing anxiety.

Two metrics matter most when evaluating a classifier's predictions. Precision answers: of everything the model flagged as positive, how many actually were positive? (If the model flags 100 transactions and 85 are real fraud, precision is 85%.) Recall answers: of everything that actually was positive, how many did the model catch? (If there are 200 fraud cases and the model catches 150, recall is 75%.) High precision means few false alarms; high recall means few missed cases. In fraud detection, a bank typically prefers high recall — it is better to investigate a few extra transactions than to let a thief steal from customers.