Probability: a reasoned summary

A compact reference for modelling uncertainty, updating information and recognising the main probability distributions.

Events and axioms

The sample space $Ω$ contains all possible outcomes; an event is a subset $A \subseteq Ω$ . A probability satisfies non-negativity, $P (Ω) = 1$ , and countable additivity for disjoint events.

P (A^{c}) = 1 - P (A)

“Favourable cases over possible cases” only applies to finite, equally likely outcomes.

Counting

Order matters

Permutations: $n!$ . Ordered selections of $k$ : $\frac{n!}{(n - k)!}$ .

Order does not matter

Combinations: $(\binom{n}{k}) = \frac{n!}{k! (n - k)!}$ .

Repetition allowed

Sequences of length $k$ over $n$ choices: $n^{k}$ .

Conditional probability and Bayes

P (A | B) = \frac{P (A \cap B)}{P (B)}

Events $A$ and $B$ are independent exactly when $P (A \cap B) = P (A) P (B)$ . Independence is not the same as mutual exclusivity.

Random variables, expectation and variance

A random variable is a function $X : Ω \to ℝ$ . Discrete variables have a probability mass function; continuous variables have a density. Both have the cumulative distribution function $F_{X} (x) = P (X \leq x)$ .

Var (X) = 𝔼 [X^{2}] - {𝔼 [X]}^{2}

Expectation is linear. Variances add for independent variables; in general the covariance term must be included.

Explore variance, standard deviation and distributions interactively →

Key distributions

Distribution	Use	Mean / variance
Bernoulli $Ber (p)$	One success/failure trial	$p / p (1 - p)$
Binomial $Bin (n, p)$	Successes in independent trials	$n p / n p (1 - p)$
Poisson $Pois (λ)$	Event counts	$λ / λ$
Exponential $Exp (λ)$	Waiting times	$\frac{1}{λ} / \frac{1}{λ^{2}}$
Normal $N (μ, σ^{2})$	Errors and sums of many effects	$μ / σ^{2}$

Law of large numbers and central limit theorem

The law of large numbers says that the sample mean converges to the population mean. The central limit theorem says that, under standard assumptions, its standardised fluctuations approach a normal distribution. These are related but distinct statements.

Before university, be comfortable translating verbal problems into events, checking independence, applying total probability and Bayes, and computing expectation and variance.