Probability spaces, σ-algebras and random variables

The lesson "Random walks and Brownian motion" used probabilities, expectations and independence freely. The continuous-time models that follow need these words to have exact meanings, because they talk about information, about events that depend on a whole path, and about averages conditional on what is known. This lesson sets up the measure-theoretic framework: the sample space, σ-algebras, probability measures and random variables. A fair die serves as the running example throughout, and the last section previews conditional expectation, the subject of the lesson after next.

Sample space and events

A random experiment is modelled by a set $\Omega$, the sample space, whose elements $\omega \in \Omega$ are the possible outcomes. For one roll of a die, $\Omega = \{1, 2, 3, 4, 5, 6\}$. For the coin-flip walk of the previous lesson with $n$ tosses, $\Omega = \{H, T\}^n$, and one $\omega$ is a whole sequence of tosses. For Brownian motion, one $\omega$ is a whole continuous path.

An event is a subset $A \subseteq \Omega$: the event occurs if the realised outcome $\omega$ lies in $A$. The event "the die shows an even number" is $\{2, 4, 6\}$. Set operations translate logic: $A \cap B$ is "$A$ and $B$", $A \cup B$ is "$A$ or $B$", the complement $A^c = \Omega \setminus A$ is "not $A$".

σ-algebras

We do not assign a probability to every subset of $\Omega$. Instead we fix a collection $\mathcal{F}$ of subsets, the events we are allowed to ask about, and require it to be closed under the logical operations. A collection $\mathcal{F}$ of subsets of $\Omega$ is a σ-algebra if

Two consequences follow at once. From (i) and (ii), $\emptyset = \Omega^c \in \mathcal{F}$. From (ii), (iii) and De Morgan's law $\bigcap_n A_n = \big(\bigcup_n A_n^c\big)^c$, countable intersections of events are events. The "σ" refers to the countable unions in (iii); they are needed for statements about infinitely many times, such as "the walk eventually exceeds 10", which is $\bigcup_{n} \{W_n > 10\}$.

On the die, three σ-algebras will recur:

• the trivial σ-algebra $\{\emptyset, \Omega\}$, which contains no information about the roll;
• the parity σ-algebra $\{\emptyset, \{1,3,5\}, \{2,4,6\}, \Omega\}$, which can tell odd from even and nothing more;
• the power set $2^{\Omega}$ of all $2^6 = 64$ subsets, which identifies the roll exactly.

Two reasons justify the restriction to a σ-algebra. The first is technical: on $\Omega = [0, 1]$ there is no way to assign a "length" to every subset that is countably additive and unchanged by translation (Vitali's construction), so uncountable sample spaces force us to leave some sets out. The second is the reason that matters in finance: a σ-algebra smaller than $2^\Omega$ models partial information. An observer whose information is the parity σ-algebra can answer "was it even?" but not "was it a 6?".

For any collection $\mathcal{C}$ of subsets, the intersection of all σ-algebras containing $\mathcal{C}$ is again a σ-algebra, the smallest one containing $\mathcal{C}$. It is written $\sigma(\mathcal{C})$ and called the σ-algebra generated by $\mathcal{C}$. On $\mathbb{R}$, the σ-algebra generated by the open intervals is the Borel σ-algebra $\mathcal{B}(\mathbb{R})$. It contains every interval, every open and closed set and everything built from them by countable operations; we use it without constructing it further.

Probability measures

A probability measure on $(\Omega, \mathcal{F})$ is a function $\mathbb{P} : \mathcal{F} \to [0, 1]$ with $\mathbb{P}(\Omega) = 1$ that is countably additive: for pairwise disjoint $A_1, A_2, \ldots \in \mathcal{F}$,

The triple $(\Omega, \mathcal{F}, \mathbb{P})$ is a probability space. The familiar rules are consequences: $\mathbb{P}(A^c) = 1 - \mathbb{P}(A)$ (since $A$ and $A^c$ are disjoint with union $\Omega$), and $A \subseteq B$ implies $\mathbb{P}(A) \le \mathbb{P}(B)$ (since $B = A \cup (B \setminus A)$). For the fair die, $\mathcal{F} = 2^\Omega$ and $\mathbb{P}(A) = |A|/6$. The same σ-algebra can carry several measures, which is how the real-world measure $\mathbb{P}$ and the risk-neutral measure $\mathbb{Q}$ coexist on one sample space.

Random variables

A random variable is a number determined by the outcome, $X : \Omega \to \mathbb{R}$, with one requirement: every question of the form "is $X \le x$?" must be an event, so that it has a probability. Formally, $X$ is $\mathcal{F}$-measurable if

This is equivalent to $X^{-1}(B) \in \mathcal{F}$ for every Borel set $B$, because the sets $B$ with $X^{-1}(B) \in \mathcal{F}$ form a σ-algebra that contains every half-line $(-\infty, x]$, and these half-lines generate $\mathcal{B}(\mathbb{R})$.

Measurability depends on the σ-algebra. On the die take $X(\omega) = \mathbf{1}\{\omega \ge 5\}$, which is 1 on $\{5, 6\}$ and 0 otherwise. Then $\{X \le 0\} = \{1,2,3,4\}$, which is not in the parity σ-algebra, so $X$ is not parity-measurable: knowing only the parity, you cannot tell whether $X = 1$.

The σ-algebra generated by a random variable

The σ-algebra generated by $X$ is the collection of all events that can be decided by observing $X$:

It is the smallest σ-algebra for which $X$ is measurable. Read it as the list of questions that $X$ answers. For $X(\omega) = \omega \bmod 2$ on the die, $\sigma(X)$ is the parity σ-algebra. The precise sense in which $\sigma(X)$ is "the information in $X$" is the Doob–Dynkin lemma: a random variable $Y$ is $\sigma(X)$-measurable if and only if $Y = f(X)$ for some Borel function $f$. Knowing $\sigma(X)$ lets you compute exactly the functions of $X$.

Finite sample spaces and partitions

When $\Omega$ is finite, every σ-algebra has a simple shape. For $\omega \in \Omega$ let $A(\omega)$ be the intersection of all events in $\mathcal{G}$ that contain $\omega$; it is a finite intersection, so $A(\omega) \in \mathcal{G}$. These sets are called the atoms of $\mathcal{G}$. Two atoms are either equal or disjoint: if $\omega' \notin A(\omega)$, some $C \in \mathcal{G}$ contains $\omega$ but not $\omega'$, so $C^c$ contains $\omega'$ and $A(\omega') \subseteq C^c$ is disjoint from $A(\omega) \subseteq C$. Hence the atoms partition $\Omega$, and every event of $\mathcal{G}$ is the union of the atoms of its points.

On a finite $\Omega$, a σ-algebra $\mathcal{G}$ is the same thing as a partition of $\Omega$ into $m$ atoms: the events of $\mathcal{G}$ are exactly the unions of atoms, so there are $2^m$ of them. A random variable is $\mathcal{G}$-measurable if and only if it is constant on each atom.

The parity σ-algebra has atoms $\{1,3,5\}$ and $\{2,4,6\}$, so $2^2 = 4$ events. The partition into pairs $\{1,2\}, \{3,4\}, \{5,6\}$ gives $2^3 = 8$ events, and $\mathbf{1}\{\omega \ge 5\}$ is measurable for it, being constant on each pair. The power set has six atoms and $2^6 = 64$ events.

Independence

Two events are independent if $\mathbb{P}(A \cap B) = \mathbb{P}(A)\,\mathbb{P}(B)$. Two σ-algebras $\mathcal{G}$ and $\mathcal{H}$ are independent if this holds for every $A \in \mathcal{G}$ and every $B \in \mathcal{H}$, and random variables $X$ and $Y$ are independent if $\sigma(X)$ and $\sigma(Y)$ are. In words: learning the answer to any question about $X$ does not change the probabilities of any question about $Y$.

Example on the fair die. Parity and $\sigma(\mathbf{1}\{\omega \ge 5\})$ are independent: the four non-trivial pairs give

and the same for the even numbers, while pairs involving $\emptyset$ or $\Omega$ hold trivially. Parity and low–high are not independent: $\mathbb{P}(\{1,3,5\} \cap \{1,2,3\}) = \tfrac13$, but $\tfrac12 \cdot \tfrac12 = \tfrac14$.

Expectation

On a finite space the expectation is the probability-weighted average, $\mathbb{E}[X] = \sum_{\omega} X(\omega)\,\mathbb{P}(\{\omega\})$. On a general space it is the Lebesgue integral $\mathbb{E}[X] = \int_\Omega X\, d\mathbb{P}$, defined first for variables taking finitely many values and then by monotone limits; it keeps the rules you know: linearity, $\mathbb{E}[\mathbf{1}_A] = \mathbb{P}(A)$, and $\mathbb{E}[XY] = \mathbb{E}[X]\,\mathbb{E}[Y]$ for independent integrable $X, Y$. For the die with $X(\omega) = \omega$,

Conditional expectation as block averages

Given a σ-algebra $\mathcal{G}$ on a finite $\Omega$ whose atoms all have positive probability, the conditional expectation $\mathbb{E}[X \mid \mathcal{G}]$ is the random variable that, on each atom $A$, equals the average of $X$ over $A$:

It is the best guess of $X$ by an observer who knows which atom occurred. It is $\mathcal{G}$-measurable, and it has the same average as $X$ on every event of $\mathcal{G}$: $\mathbb{E}\big[\mathbb{E}[X \mid \mathcal{G}]\,\mathbf{1}_A\big] = \mathbb{E}[X\,\mathbf{1}_A]$ for $A \in \mathcal{G}$. These two properties are the definition used on general spaces, which the lesson "Conditional expectation and martingales" takes up. Averaging inside blocks removes variation, and the law of total variance makes this exact:

Worked example with $X(\omega) = \omega$. With parity information, $\mathbb{E}[X \mid \mathcal{G}]$ is $3$ on the odd numbers and $4$ on the even ones, so its variance is $0.25$. With low–high it is $2$ or $5$, variance $2.25$. With pairs it is $1.5$, $3.5$ or $5.5$, variance $8/3 \approx 2.667$. With full information it is $X$ itself, variance $35/12$; with no information it is the constant $3.5$, variance $0$.

The figure shows the six outcomes as tiles coloured by the atoms of the chosen information $\mathcal{G}$, each tile listing $\omega$, $X(\omega)$ and the block average $\mathbb{E}[X \mid \mathcal{G}](\omega)$. Choose the information (none, parity, low–high, pairs, full) and the random variable ($\omega$, $\omega^2$ or $\mathbf{1}\{\omega \ge 5\}$). The readouts give the number of events $2^{\text{blocks}}$, $\mathbb{E}[X]$, $\operatorname{Var}(X)$ and $\operatorname{Var}(\mathbb{E}[X \mid \mathcal{G}])$. Check that the last never exceeds $\operatorname{Var}(X)$ and equals it with full information. With $\mathbf{1}\{\omega \ge 5\}$ and parity it is $0$: the information is independent of $X$ and does not improve the guess.

Summary

• A probability space $(\Omega, \mathcal{F}, \mathbb{P})$ consists of outcomes, a σ-algebra of events closed under complements and countable unions, and a countably additive measure with $\mathbb{P}(\Omega) = 1$.
• A random variable is an $\mathcal{F}$-measurable map: every event $\{X \le x\}$ belongs to $\mathcal{F}$. Its generated σ-algebra $\sigma(X)$ collects the questions $X$ answers, and $\sigma(X)$-measurable variables are the functions of $X$.
• On a finite $\Omega$ a σ-algebra is a partition into $m$ atoms with $2^m$ events, and measurable means constant on atoms.
• Independence of σ-algebras means every event of one is independent of every event of the other.
• Conditional expectation on a partition is the block average; it is a coarser variable with $\operatorname{Var}(\mathbb{E}[X \mid \mathcal{G}]) \le \operatorname{Var}(X)$.

The next lesson, "Filtrations, adapted processes and stopping times", lets the σ-algebra grow with time to model information that accumulates as prices are observed.