REFERENCE SHEET

Gradients, Hessians & change of variables

Local geometry is encoded by first and second derivatives.

Before this Derivatives & Taylor expansion, Eigenvalues & diagonalization

D f (x) [h] = \nabla f (x) \cdot h H_{i j} = \frac{\partial^{2} f}{\partial x_{i} \partial x_{j}}

Definitions

Gradient

For a differentiable scalar function on Euclidean space, the vector representing its derivative under the Euclidean inner product.

Jacobian

The matrix of first partial derivatives of a differentiable vector-valued map.

Hessian

The matrix of second partial derivatives of a scalar function; symmetric when the second partials are continuous.

Key theorems

Second derivative test

At a critical point of a C² function, a positive-definite Hessian gives a strict local minimum, negative-definite a strict local maximum, and indefinite a saddle. Semidefinite cases are inconclusive.

Change of variables

For a C¹ diffeomorphism φ between open regions and an integrable function, integration transforms using |det Dφ| as the volume factor.

Lagrange multipliers

At a constrained local extremum of differentiable f under g(x)=c, if ∇g≠0, then ∇f=λ∇g for some λ.

Notable applications

Optimization

Gradients guide descent and Hessians describe local curvature.

Density transformations

The Jacobian determinant converts probability densities between coordinate systems.

Work it out

Try before revealing

01 / Classify

Classify the origin for f(x,y)=x²−y².

Hint

Find the Hessian eigenvalues.

Solution

∇f(0,0)=0 and H=diag(2,−2), so the origin is a saddle.

02 / Compute

Compute the area of a disk of radius R using polar coordinates.

Hint

The Jacobian factor is r.