Introduction to Probability
Dr. Mine Dogucu
Random process
In random processes there is more than one possible outcome and the deterministic prediction of the outcome is difficult.
e.g. flip of a coin
e.g. roll of a die
Sample Space
Sample space is the set of all possible outcomes of a random process.
Examples
In a single roll of a die:
\(S = \{ 1, 2, 3, 4, 5, 6\}\)
In flipping a single coin twice:
\(S = \{ HH, TT, HT, TH\}\)
Event
An event is a subset of the sample space.
Examples
Let A represent the event that a single roll die results in an even number.
\(A = \{2, 4, 6 \}\)
Let B represent the event that a single roll die results in an odd number.
\(B = \{1, 3, 5 \}\)
Let C represent the event that a single roll die results in a prime number.
\(C = \{2, 3, 5 \}\)
Venn Diagram
Complement of an Event
Definition Complement of an event is the set of all outcomes in the sample space that are not in the event itself.
Example
Complement of event \(C\) is the set of outcomes in a roll die that are not prime.
Notation \(C^c\)
Pronunciation C complement
\(C^c = \{1, 4, 6\}\)
Mutually Exclusive or Disjoint Events
Definition Mutually exclusive or disjoint events are two events that cannot happen at the same time.
Examples
Events A and B are mutually exclusive because an outcome of a roll of a die cannot be even and odd.
Events A and C are not mutually exclusive because 2 is both an even number and a prime number so event A and C can happen simultaneously.
Events and Set Notation
| Description | Notation | Reading | Elements |
|---|---|---|---|
| Union | \(A \cup C\) | A or C | {2, 3, 4, 5, 6} |
| Intersection | \(A \cap C\) | A and C | {2} |
Defining Probability
Frequentist definition
The probability of an outcome is defined to be the proportion of times the outcome is observed under high number of repetitions of the random process.
Assume that we are repeating the random process of a coin flip and are recording \(X\), the number of heads in \(n\) coin flips. Then
\[P(H) = \lim_{n\to\infty}\frac{X}{n}\]
\[P(H) =\frac{1}{2}\]
Defining Probability
Bayesian definition
When defining probability in addition to considering number of times an outcome occurs, Bayesians consider prior information that they have about an outcome.
More on this in Stats 115.
Axioms of Probability
- The probability of any event is between zero and one:
\[0 \leq P(A) \leq 1\]
- The probabilities must add up to 1.
\[P(S) = 1\]
- The probability of mutually exclusive events is additive.
\[\bigcup\limits_{i=1}^{\infty} A_{i} = \Sigma_{i =1}^\infty P(A_i)\]
Probability of Complementary Events
\(P(A) + P(A^c) = 1\)
Example
If we know the the probability that somebody owns a bike is 0.08, then we would know that the probability that somebody does not own a bike is 0.92.
Independence and Multiplication Rule
Two processes are independent if knowing about the outcome of one does not help predict the outcome of the other.
Example
two flips of a single coin
If events A and B are from two independent processes, then
\[P(A \cap B) = P(A) \times P(B)\]
Probability of getting two heads in two flips of a single coin is
\(\frac{1}{2} \times \frac{1}{2} = \frac{1}{4}\)
Data - GSS 2018
The General Social Survey (GSS) is a sociological survey that has been regularly conducted since 1972. It is a comprehensive survey that provides information on experiences of residents of the United States.
| Belief in Life After Death | ||||
|---|---|---|---|---|
| Yes |
No |
Total |
||
|
College Science Course |
Yes | 375 | 75 | 450 |
| No | 485 | 115 | 600 | |
| Total | 860 | 190 | 1050 | |
Events
Let B represent an event that a randomly selected person in this sample believes in life after death.
Let C represent an event that a randomly selected person in this sample took a college level science course.
Joint Probability
Note that events \(B\) and \(C\) are not mutually exclusive. A randomly selected person can believe in life after death and might have taken a college science course. \(B \cap C \neq \emptyset\).
\(P(B \cap C) = \frac{375}{1050}\)
Note that \(P(B\cap C) = P(C\cap B)\). Order does not matter.
Marginal Probability
\(P(B)\) represents a marginal probability. So do \(P(C)\), \(P(B^C)\) and \(P(C^C)\). In order to calculate these probabilities we could only use the values in the margins of the contingency table, hence the name.
\(P(B) = \frac{860}{1050}\)
\(P(C) = \frac{450}{1050}\)
Conditional Probability
\(P(B|C)\) represents a conditional probability. So do \(P(B^c|C)\), \(P(C| B)\) and \(P(C|B^c)\). To calculate the probabilities we focus on the row or the column of the given information. We reduce the sample space to this given information.
Probability that a randomly selected person believes in life after death given that they have taken a college science course
\(P(B|C) = \frac{375}{450}\)
Conditional Probability
The order matters!
\(P(\text{has a dog | like dogs}) \neq\) \(P(\text{like dogs | has a dog})\)
Addition Rule
\(P(B\cup C) = P(B) + P(C) - P(B \cap C)\)
\(P(B\cup C) = \frac{860}{1050} + \frac{450}{1050} -\frac{375}{1050} = \frac{935}{1050}\)
\(P(B\cup C) = \frac{375}{1050} + \frac{75}{1050} +\frac{485}{1050} = \frac{935}{1050}\)
