Agenda¶
We'll cover the basics of probability theory. This is a math lesson; take written notes ✍🏽.
Probability resources¶
Probability is a tricky subject. If it doesn't click during lecture or on the assignments, take a look at the following resources:
Probability theory¶
- Some things in life seem random.
- e.g., flipping a coin or rolling a die 🎲.
- The probability of seeing "heads" when flipping a fair coin is $\frac{1}{2}$.
- One interpretation of probability says that if we flipped a coin infinitely many times, then $\frac{1}{2}$ of the outcomes would be heads.
Terminology¶
- Experiment: A process or action whose result is random.
- e.g., rolling a die.
- e.g., flipping a coin twice.
- Outcome: The result of an experiment.
- e.g., the possible outcomes of rolling a six-sided die are 1, 2, 3, 4, 5, and 6.
- e.g., the possible outcomes of flipping a coin twice are HH, HT, TH, and TT.
- Event: A set of outcomes.
- e.g., the event that the die lands on a even number is the set of outcomes {2, 4, 6}.
- e.g., the event that the die lands on a 5 is the set of outcomes {5}.
- e.g., the event that there is at least 1 head in 2 flips is the set of outcomes {HH, HT, TH}.
Terminology¶
- Probability: A number between 0 and 1 (equivalently, between 0% and 100%) that describes the likelihood of an event.
- 0: The event never happens.
- 1: The event always happens.
- Notation: If $A$ is an event, $P(A)$ is the probability of that event.
Equally-likely outcomes¶
- If all of the possible outcomes are equally likely, then the probability of $A$ is
$$ P(A) = \frac{ \# \text{ of outcomes satisfying } A }{ \text{total } \# \text{ of outcomes} } $$
- Example 1: Suppose we flip a fair coin 3 times. What is the probability we see exactly 2 heads?
Example 1 solved¶
- When we flip a fair coin 3 times, there are 8 possible outcomes: HHH, HHT, HTH, HTT, THH, THT, TTH, and TTT.
- These outcomes are all equally likely.
- 3 of these outcomes have exactly 2 heads: HHT, HTH, and THH.
- So, the probability of seeing exactly 2 heads in 3 flips of a fair coin is $\frac{3}{8}$.
Concept Check ✅¶
I have three cards: red, blue, and green. What is the chance that I choose a card at random and it is green, then – without putting it back – I choose another card at random and it is red?
- A) $\frac{1}{9}$
- B) $\frac{1}{6}$
- C) $\frac{1}{3}$
- D) $\frac{2}{3}$
- E) None of the above.
Solution¶
- There are 6 possible outcomes: RG, RB, GR, GB, BR, and BG.
- These outcomes are equally likely.
- There is only 1 outcome which makes the event happen: GR.
- Hence the probability is $\frac{1}{6}$.
Conditional probabilities¶
- Two events $A$ and $B$ can both happen. Suppose that we know $A$ has happened, but we don't know if $B$ has.
- If all outcomes are equally likely, then the conditional probability of $B$ given $A$ is:
$$ P(B \text{ given } A) = \frac{ \# \text{ of outcomes satisfying both } A \text{ and } B }{ \# \text{ of outcomes satisfying } A } $$
- Intuitively, this is similar to the definition of the regular probability of $B$:
$$ P(B) = \frac{ \# \text{ of outcomes satisfying } B }{ \text{total } \# \text{ of outcomes} } $$
if you restrict the set of possible outcomes to just those in event $A$.
Concept Check ✅¶
$$ P(B \text{ given } A) = \frac{ \# \text{ of outcomes satisfying both } A \text{ and } B }{ \# \text{ of outcomes satisfying } A } $$
I roll a six-sided die and don't tell you what the result is, but I tell you that it is 3 or less. What is the probability that the result is even?
- A) $\frac{1}{2}$
- B) $\frac{1}{3}$
- C) $\frac{1}{4}$
- D) None of the above.
Solution¶
$$ P(B \text{ given } A) = \frac{ \# \text{ of outcomes satisfying both } A \text{ and } B }{ \# \text{ of outcomes satisfying } A } $$
- $A$ is the event "roll is 3 or less", $B$ is the event "roll is even".
- There are three outcomes where the roll is 3 or less: 1, 2, and 3.
- There is only one outcome where the roll is 3 or less and even: 2.
- So the probability that the roll is even given that it is 3 or less is $P(B \text{ given } A) = \frac{1}{3}$.
Probability that two events both happen¶
- Suppose again that $A$ and $B$ are two events, and that all outcomes are equally likely. Then, the probability that both $A$ and $B$ occur is
$$ P(A \text{ and } B) = \frac{ \# \text{ of outcomes satisfying both } A \text{ and } B }{ \text{total } \# \text{ of outcomes} } $$
- Example 2: I roll a fair six-sided die. What is the probability that the roll is 3 or less and even?
Example 2 solved¶
I roll a fair six-sided die. What is the probability that the roll is 3 or less and even?
- Only one outcome is both 3 or less and even: 2.
- There are 6 total outcomes.
- Thus, $P(A \text{ and } B) = \frac{1}{6}$.
The multiplication rule¶
- The multiplication rule specifies how to compute the probability of both $A$ and $B$ happening, even if all outcomes are not equally likely.
$$ P(A \text{ and } B) = P(A) \cdot P(B \text{ given } A) $$
- Example 2, again: I roll a fair six-sided die. What is the probability that the roll is 3 or less and even?
Example 2 solved, again¶
I roll a fair six-sided die. What is the probability that the roll is 3 or less and even?
- The probability that the roll is 3 or less is $P(A) = \frac{1}{2}$.
- From before, the probability that the roll is even given that the roll is 3 or less is $P(B \text{ given } A) = \frac{1}{3}$.
- Thus, the probability the roll is both 3 or less and even is $P(A \text{ and } B) = \frac{1}{2} \cdot \frac{1}{3} = \frac{1}{6}$.
- Note that an equivalent formula is $P(A \text{ and } B) = P(B) \cdot P (A \text{ given } B)$.
Generally, situations involving an "and" involve multiplication.¶
What if $A$ isn't affected by $B$? 🤔¶
- The multiplication rule states that, for any two events $A$ and $B$, $$P(A \text{ and } B) = P(A) \cdot P(B \text{ given } A)$$
- What if knowing that $A$ happens doesn't tell you anything about the likelihood of $B$ happening?
- Suppose we flip a fair coin three times.
- The probability that the second flip is heads doesn't depend on the result of the first flip.
- Then, what is $P(A \text{ and } B)$?
Independent events¶
- Two events $A$ and $B$ are independent if $P(B \text{ given } A) = P(B) \:$, or equivalently if $$P(A \text{ and } B) = P(A) \cdot P(B)$$
- Example 3: Suppose we have a coin that is biased, and flips heads with probability 0.7. Each flip is independent of all other flips. We flip it 5 times. What's the probability we see 5 heads in a row?
Example 3 solved¶
Suppose we have a coin that is biased, and flips heads with probability 0.7. Each flip is independent of all other flips. We flip it 5 times. What's the probability we see 5 heads in a row?
- The probability of seeing heads on a single flip is 0.7.
- Each flip is independent.
- So, the probability of seeing 5 heads in a row is
$$0.7 \cdot 0.7 \cdot 0.7 \cdot 0.7 \cdot 0.7 = 0.7^5$$
Probability that an event doesn't happen¶
- The probability that $A$ doesn't happen is $1 - P(A) \:$.
- For example, if the probability it is sunny tomorrow is 0.85, then the probability it is not sunny tomorrow is 0.15.
Concept Check ✅¶
Every time I call my grandma 👵, the probability that she answers her phone is $\frac{1}{3}$, independently for each call. If I call my grandma three times today, what is the chance that I will talk to her at least once?
- A) $\frac{1}{3}$
- B) $\frac{2}{3}$
- C) $\frac{1}{2}$
- D) $1$
- E) None of the above.
Solution¶
- Let's first calculate the probability that she doesn't answer her phone at all, on any of the three calls.
- The probability she doesn't answer her phone on any one attempt is $\frac{2}{3}$.
- So the probability she doesn't answer her phone in three tries is $\frac{2}{3} \cdot \frac{2}{3} \cdot \frac{2}{3} = \frac{8}{27}$.
- But we want the probability of her answering at least once. So we subtract the above result from 1.
- $1 - \frac{8}{27} = \frac{19}{27}$; none of the above!
Probability of either of two events happening¶
- Suppose again that $A$ and $B$ are two events, and that all outcomes are equally likely. Then, the probability that either $A$ or $B$ occur is
$$ P(A \text{ or } B) = \frac{ \# \text{ of outcomes satisfying either } A \text{ or } B }{ \text{total } \# \text{ of outcomes} } $$
- Example 4: I roll a fair six-sided die. What is the probability that the roll is even or at least 5?
Example 4 solved¶
I roll a fair six-sided die. What is the probability that the roll is even or at least 5?
- There are three outcomes that are even: 2, 4, 6.
- There are two outcomes that are at least 5: 5, 6.
- There are four total outcomes that satisfy at least one of the two conditions: 2, 4, 5, 6.
- Thus, the probability that the roll is even or at least 5 is $\frac{4}{6} = \frac{2}{3}$.
- Note that this is not $P(A) + P(B) \:$, which would be $\frac{3}{6} + \frac{2}{6} = \frac{5}{6}$, because there is overlap between events $A$ and $B$.
The addition rule¶
- Suppose that if $A$ happens, then $B$ doesn't, and if $B$ happens, then $A$ doesn't.
- Such events are called mutually exclusive – they have no overlap.
- If $A$ and $B$ are any two mutually exclusive events, then
$$P(A \text{ or } B) = P(A) + P(B)$$
- Example 5: Suppose I have two biased coins, coin $A$ and coin $B$. Coin $A$ flips heads with probability 0.6, and coin $B$ flips heads with probability 0.3. I flip both coins once. What's the probability I see two different faces?
Example 5 solved¶
Suppose I have two biased coins, coin $A$ and coin $B$. Coin $A$ flips heads with probability 0.6, and coin $B$ flips heads with probability 0.3. The two coins are independent of one another. I flip both coins once. What's the probability I see two different faces?
- The event we see two different faces corresponds to either seeing a head then a tail, or a tail then a head (i.e. not both heads and not both tails).
- The probability of seeing a head then a tail is $0.6 \cdot (1 - 0.3)$, because the two coins are independent of one another.
- The probability of seeing a tail then a head is $(1 - 0.6) \cdot 0.3$.
- So, the probability of seeing two different faces is
$$0.6 \cdot (1 - 0.3) + (1 - 0.6) \cdot 0.3 = 0.54$$
Generally, situations involving an "or" involve addition.¶
Aside: Proof of the addition rule for equally-likely events¶
You are not required to know how to "prove" anything in this course; you may just find this interesting.
If $A$ and $B$ are events consisting of equally likely outcomes, and furthermore $A$ and $B$ are mutually exclusive (meaning they have no overlap), then
$$ \begin{align*} P(A \text{ or } B) &= \frac{ \# \text{ of outcomes satisfying either } A \text{ or } B }{ \text{total } \# \text{ of outcomes} } \\[1em] &= \frac{ (\# \text{ of outcomes satisfying } A) + (\# \text{ of outcomes satisfying } B) }{ \text{total } \# \text{ of outcomes} } \\[1em] &= \frac{ (\# \text{ of outcomes satisfying } A) }{ \text{total } \# \text{ of outcomes} } + \frac{ (\# \text{ of outcomes satisfying } B) }{ \text{total } \# \text{ of outcomes} } \\[1em] &= P(A) + P(B) \end{align*} $$
Summary, next time¶
- Probability describes the likelihood of an event occurring.
- There are several rules for computing probabilities. We looked at many special cases that involved equally-likely events.
- There are two general rules to be aware of:
- The multiplication rule, which states that for any two events, $P(A \text{ and } B) = P(B \text{ given } A) \cdot P(A) \:$.
- The addition rule, which states that for any two mutually exclusive events, $P(A \text{ or } B) = P(A) + P(B)$.
- Next time: Simulations.