Information
A sub-σ-algebra 𝒢⊆ℱ records which events are observable. E[X|Y] abbreviates E[X|σ(Y)].
Find a concept
REFERENCE SHEET
Conditional expectation
Average using the information available; least squares and conditioning are versions of the same projection idea.
Before this Events, conditioning & Bayes, Lebesgue integration & Lᵖ spaces, Orthogonality, least squares & SVD
E[E[X | 𝒢]] = E[X]
Definitions
Conditional expectation
For integrable real X, E[X|𝒢] is a 𝒢-measurable integrable random variable Z satisfying ∫ₐZ dP=∫ₐX dP for every A∈𝒢. It exists and is unique up to almost sure equality.
Finite partition
If 𝒢 is generated by a finite partition with positive-probability cells Aᵢ, then E[X|𝒢] equals E[X1ₐᵢ]/P(Aᵢ) on each cell.
Key theorems
Tower property
If ℋ⊆𝒢⊆ℱ and X is integrable, E[E[X|𝒢]|ℋ]=E[X|ℋ] almost surely. In particular, taking total expectations recovers E[X].
Taking out known factors
For bounded 𝒢-measurable Y and integrable X, E[YX|𝒢]=Y E[X|𝒢] almost surely.
Least-squares projection
If X∈L², then E[X|𝒢] uniquely minimizes E[(X−Z)²] among 𝒢-measurable Z∈L², up to almost sure equality. The residual is orthogonal to every such Z.
Notable applications
Prediction
E[X|Y] is the best squared-error predictor of X from Y. Ordinary linear regression restricts predictions to linear combinations of features; the conditional mean need not be linear.
Variance decomposition
For X∈L², Var(X)=E[Var(X|𝒢)]+Var(E[X|𝒢]). This separates uncertainty remaining after observation from variation explained by it.
Work it out
Try before revealing01 / Compute
A fair die gives X. You learn only whether X is odd. Find E[X|parity] and verify its expectation.
Hint
Average separately on {1,3,5} and {2,4,6}.
Solution
The conditional mean is 3 on odd outcomes and 4 on even outcomes. Its expectation is (3+4)/2=3.5=E[X].
02 / Prove
Write m=E[X|𝒢] for X∈L². Show E[(X−Z)²]=E[(X−m)²]+E[(m−Z)²] for any 𝒢-measurable Z∈L².
Hint
Expand the square and use residual orthogonality.
Solution
Write X−Z=(X−m)+(m−Z). The cross term has expectation zero because m−Z is 𝒢-measurable and X−m is orthogonal to L²(𝒢). The two remaining squares give the identity, proving minimality and uniqueness up to almost sure equality.
Connect the ideas
Go deeper
Further reading and full-length treatments: