Lecture 7 – Probability¶

DSC 10, Summer 2025¶

Agenda¶

We'll cover the basics of probability theory. This is a math lesson; take written notes ✍🏽.

Probability resources¶

Probability is a tricky subject. If it doesn't click during lecture or on the assignments, take a look at the following resources:

  • Computational and Inferential Thinking, Chapter 9.5.
  • Theory Meets Data, Chapters 1 and 2.
  • Khan Academy's unit on Probability.

Probability theory¶

  • Some things in life seem random.
    • e.g., flipping a coin or rolling a die 🎲.
  • The probability of seeing "heads" when flipping a fair coin is $\frac{1}{2}$.
  • One interpretation of probability says that if we flipped a coin infinitely many times, then $\frac{1}{2}$ of the outcomes would be heads.

Terminology¶

  • Experiment: A process or action whose result is random.
    • e.g., rolling a die.
    • e.g., flipping a coin twice.
  • Outcome: The result of an experiment.
    • e.g., the possible outcomes of rolling a six-sided die are 1, 2, 3, 4, 5, and 6.
    • e.g., the possible outcomes of flipping a coin twice are HH, HT, TH, and TT.
  • Event: A set of outcomes.
    • e.g., the event that the die lands on a even number is the set of outcomes {2, 4, 6}.
    • e.g., the event that the die lands on a 5 is the set of outcomes {5}.
    • e.g., the event that there is at least 1 head in 2 flips is the set of outcomes {HH, HT, TH}.

Terminology¶

  • Probability: A number between 0 and 1 (equivalently, between 0% and 100%) that describes the likelihood of an event.
    • 0: The event never happens.
    • 1: The event always happens.
  • Notation: If $A$ is an event, $P(A)$ is the probability of that event.

Equally-likely outcomes¶

  • If all of the possible outcomes are equally likely, then the probability of $A$ is

$$ P(A) = \frac{ \# \text{ of outcomes satisfying } A }{ \text{total } \# \text{ of outcomes} } $$

  • Example 1: Suppose we flip a fair coin 3 times. What is the probability we see exactly 2 heads?

Example 1 solved¶

  • When we flip a fair coin 3 times, there are 8 possible outcomes: HHH, HHT, HTH, HTT, THH, THT, TTH, and TTT.
  • These outcomes are all equally likely.
  • 3 of these outcomes have exactly 2 heads: HHT, HTH, and THH.
  • So, the probability of seeing exactly 2 heads in 3 flips of a fair coin is $\frac{3}{8}$.

Concept Check ✅¶

I have three cards: red, blue, and green. What is the chance that I choose a card at random and it is green, then – without putting it back – I choose another card at random and it is red?

  • A) $\frac{1}{9}$
  • B) $\frac{1}{6}$
  • C) $\frac{1}{3}$
  • D) $\frac{2}{3}$
  • E) None of the above.

Solution¶

  • There are 6 possible outcomes: RG, RB, GR, GB, BR, and BG.
  • These outcomes are equally likely.
  • There is only 1 outcome which makes the event happen: GR.
  • Hence the probability is $\frac{1}{6}$.

Conditional probabilities¶

  • Two events $A$ and $B$ can both happen. Suppose that we know $A$ has happened, but we don't know if $B$ has.
  • If all outcomes are equally likely, then the conditional probability of $B$ given $A$ is:

$$ P(B \text{ given } A) = \frac{ \# \text{ of outcomes satisfying both } A \text{ and } B }{ \# \text{ of outcomes satisfying } A } $$

  • Intuitively, this is similar to the definition of the regular probability of $B$:

$$ P(B) = \frac{ \# \text{ of outcomes satisfying } B }{ \text{total } \# \text{ of outcomes} } $$

if you restrict the set of possible outcomes to just those in event $A$.

Concept Check ✅¶

$$ P(B \text{ given } A) = \frac{ \# \text{ of outcomes satisfying both } A \text{ and } B }{ \# \text{ of outcomes satisfying } A } $$

I roll a six-sided die and don't tell you what the result is, but I tell you that it is 3 or less. What is the probability that the result is even?

  • A) $\frac{1}{2}$
  • B) $\frac{1}{3}$
  • C) $\frac{1}{4}$
  • D) None of the above.

Solution¶

$$ P(B \text{ given } A) = \frac{ \# \text{ of outcomes satisfying both } A \text{ and } B }{ \# \text{ of outcomes satisfying } A } $$

  • $A$ is the event "roll is 3 or less", $B$ is the event "roll is even".
  • There are three outcomes where the roll is 3 or less: 1, 2, and 3.
  • There is only one outcome where the roll is 3 or less and even: 2.
  • So the probability that the roll is even given that it is 3 or less is $P(B \text{ given } A) = \frac{1}{3}$.

Probability that two events both happen¶

  • Suppose again that $A$ and $B$ are two events, and that all outcomes are equally likely. Then, the probability that both $A$ and $B$ occur is

$$ P(A \text{ and } B) = \frac{ \# \text{ of outcomes satisfying both } A \text{ and } B }{ \text{total } \# \text{ of outcomes} } $$

  • Example 2: I roll a fair six-sided die. What is the probability that the roll is 3 or less and even?

Example 2 solved¶

I roll a fair six-sided die. What is the probability that the roll is 3 or less and even?

  • Only one outcome is both 3 or less and even: 2.
  • There are 6 total outcomes.
  • Thus, $P(A \text{ and } B) = \frac{1}{6}$.

The multiplication rule¶

  • The multiplication rule specifies how to compute the probability of both $A$ and $B$ happening, even if all outcomes are not equally likely.

$$ P(A \text{ and } B) = P(A) \cdot P(B \text{ given } A) $$

  • Example 2, again: I roll a fair six-sided die. What is the probability that the roll is 3 or less and even?

Example 2 solved, again¶

I roll a fair six-sided die. What is the probability that the roll is 3 or less and even?

  • The probability that the roll is 3 or less is $P(A) = \frac{1}{2}$.
  • From before, the probability that the roll is even given that the roll is 3 or less is $P(B \text{ given } A) = \frac{1}{3}$.
  • Thus, the probability the roll is both 3 or less and even is $P(A \text{ and } B) = \frac{1}{2} \cdot \frac{1}{3} = \frac{1}{6}$.
  • Note that an equivalent formula is $P(A \text{ and } B) = P(B) \cdot P (A \text{ given } B)$.

Generally, situations involving an "and" involve multiplication.¶

What if $A$ isn't affected by $B$? 🤔¶

  • The multiplication rule states that, for any two events $A$ and $B$, $$P(A \text{ and } B) = P(A) \cdot P(B \text{ given } A)$$
  • What if knowing that $A$ happens doesn't tell you anything about the likelihood of $B$ happening?
    • Suppose we flip a fair coin three times.
    • The probability that the second flip is heads doesn't depend on the result of the first flip.
  • Then, what is $P(A \text{ and } B)$?

Independent events¶

  • Two events $A$ and $B$ are independent if $P(B \text{ given } A) = P(B) \:$, or equivalently if $$P(A \text{ and } B) = P(A) \cdot P(B)$$
  • Example 3: Suppose we have a coin that is biased, and flips heads with probability 0.7. Each flip is independent of all other flips. We flip it 5 times. What's the probability we see 5 heads in a row?

Example 3 solved¶

Suppose we have a coin that is biased, and flips heads with probability 0.7. Each flip is independent of all other flips. We flip it 5 times. What's the probability we see 5 heads in a row?

  • The probability of seeing heads on a single flip is 0.7.
  • Each flip is independent.
  • So, the probability of seeing 5 heads in a row is

$$0.7 \cdot 0.7 \cdot 0.7 \cdot 0.7 \cdot 0.7 = 0.7^5$$

Probability that an event doesn't happen¶

  • The probability that $A$ doesn't happen is $1 - P(A) \:$.
  • For example, if the probability it is sunny tomorrow is 0.85, then the probability it is not sunny tomorrow is 0.15.

Concept Check ✅¶

Every time I call my grandma 👵, the probability that she answers her phone is $\frac{1}{3}$, independently for each call. If I call my grandma three times today, what is the chance that I will talk to her at least once?

  • A) $\frac{1}{3}$
  • B) $\frac{2}{3}$
  • C) $\frac{1}{2}$
  • D) $1$
  • E) None of the above.

Solution¶

  • Let's first calculate the probability that she doesn't answer her phone at all, on any of the three calls.
    • The probability she doesn't answer her phone on any one attempt is $\frac{2}{3}$.
    • So the probability she doesn't answer her phone in three tries is $\frac{2}{3} \cdot \frac{2}{3} \cdot \frac{2}{3} = \frac{8}{27}$.
  • But we want the probability of her answering at least once. So we subtract the above result from 1.
    • $1 - \frac{8}{27} = \frac{19}{27}$; none of the above!

Probability of either of two events happening¶

  • Suppose again that $A$ and $B$ are two events, and that all outcomes are equally likely. Then, the probability that either $A$ or $B$ occur is

$$ P(A \text{ or } B) = \frac{ \# \text{ of outcomes satisfying either } A \text{ or } B }{ \text{total } \# \text{ of outcomes} } $$

  • Example 4: I roll a fair six-sided die. What is the probability that the roll is even or at least 5?

Example 4 solved¶

I roll a fair six-sided die. What is the probability that the roll is even or at least 5?

  • There are three outcomes that are even: 2, 4, 6.
  • There are two outcomes that are at least 5: 5, 6.
  • There are four total outcomes that satisfy at least one of the two conditions: 2, 4, 5, 6.
  • Thus, the probability that the roll is even or at least 5 is $\frac{4}{6} = \frac{2}{3}$.
    • Note that this is not $P(A) + P(B) \:$, which would be $\frac{3}{6} + \frac{2}{6} = \frac{5}{6}$, because there is overlap between events $A$ and $B$.

The addition rule¶

  • Suppose that if $A$ happens, then $B$ doesn't, and if $B$ happens, then $A$ doesn't.
    • Such events are called mutually exclusive – they have no overlap.
  • If $A$ and $B$ are any two mutually exclusive events, then

$$P(A \text{ or } B) = P(A) + P(B)$$

  • Example 5: Suppose I have two biased coins, coin $A$ and coin $B$. Coin $A$ flips heads with probability 0.6, and coin $B$ flips heads with probability 0.3. I flip both coins once. What's the probability I see two different faces?

Example 5 solved¶

Suppose I have two biased coins, coin $A$ and coin $B$. Coin $A$ flips heads with probability 0.6, and coin $B$ flips heads with probability 0.3. The two coins are independent of one another. I flip both coins once. What's the probability I see two different faces?

  • The event we see two different faces corresponds to either seeing a head then a tail, or a tail then a head (i.e. not both heads and not both tails).
  • The probability of seeing a head then a tail is $0.6 \cdot (1 - 0.3)$, because the two coins are independent of one another.
  • The probability of seeing a tail then a head is $(1 - 0.6) \cdot 0.3$.
  • So, the probability of seeing two different faces is

$$0.6 \cdot (1 - 0.3) + (1 - 0.6) \cdot 0.3 = 0.54$$

Generally, situations involving an "or" involve addition.¶

Aside: Proof of the addition rule for equally-likely events¶

You are not required to know how to "prove" anything in this course; you may just find this interesting.

If $A$ and $B$ are events consisting of equally likely outcomes, and furthermore $A$ and $B$ are mutually exclusive (meaning they have no overlap), then

$$ \begin{align*} P(A \text{ or } B) &= \frac{ \# \text{ of outcomes satisfying either } A \text{ or } B }{ \text{total } \# \text{ of outcomes} } \\[1em] &= \frac{ (\# \text{ of outcomes satisfying } A) + (\# \text{ of outcomes satisfying } B) }{ \text{total } \# \text{ of outcomes} } \\[1em] &= \frac{ (\# \text{ of outcomes satisfying } A) }{ \text{total } \# \text{ of outcomes} } + \frac{ (\# \text{ of outcomes satisfying } B) }{ \text{total } \# \text{ of outcomes} } \\[1em] &= P(A) + P(B) \end{align*} $$

Summary, next time¶

  • Probability describes the likelihood of an event occurring.
  • There are several rules for computing probabilities. We looked at many special cases that involved equally-likely events.
  • There are two general rules to be aware of:
    • The multiplication rule, which states that for any two events, $P(A \text{ and } B) = P(B \text{ given } A) \cdot P(A) \:$.
    • The addition rule, which states that for any two mutually exclusive events, $P(A \text{ or } B) = P(A) + P(B)$.
  • Next time: Simulations.