3.11  Determinant and inverse

The determinant as an area factor

The columns of a \(2\times 2\) matrix \(\bm A = [\bm a_1\ \bm a_2]\) are the images of \(\bm i\) and \(\bm j\). The unit square spanned by \(\bm i\) and \(\bm j\) therefore lands on the parallelogram spanned by \(\bm a_1\) and \(\bm a_2\), and in Chapter 3.7 we found the signed area of that parallelogram to be the determinant of the matrix with the two vectors as columns,

\[ \boxed{\det\bm A = \det\begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22}\end{bmatrix} = a_{11}a_{22} - a_{12}a_{21}} \tag{3.11.1}\]

The unit square has area \(1\), so its image has area \(|\det\bm A|\). The same holds for every other region. A linear map keeps the grid lines straight, parallel and evenly spaced, so each small square of the grid lands on a copy of one and the same parallelogram, scaled by the same factor. A region covered by many small squares, a disk say, has its area multiplied by that factor too.

The video plays the matrix \(\bm A = \begin{bmatrix} 3 & 1\\ 1 & 2\end{bmatrix}\) on the plane. The unit square and a disk to its left are mapped together, and the ratio of the areas of the disk and its image, computed from the mapped outline, matches \(\det\bm A = 5\) at every instant.

Sign: the plane turned over

The determinant is a signed area. It is positive when the turn from \(\bm a_1\) to \(\bm a_2\) is counterclockwise, as the turn from \(\bm i\) to \(\bm j\) is, and negative when it is clockwise. A negative determinant means that the map turns the plane over, as a mirror does, while \(|\det\bm A|\) still gives the area factor. Below, the image of \(\bm j\) swings down through the line of \(\bm i\). At the crossing the square is flat and \(\det\bm A = 0\); beyond it the square reappears turned over and the determinant is negative.

Zero: the plane collapses

When \(\det\bm A = 0\) the parallelogram of the columns has no area, so \(\bm a_1\) and \(\bm a_2\) are parallel (Chapter 3.6). Every \(\bm A\bm x = x_1\bm a_1 + x_2\bm a_2\) is then a multiple of the same direction, and the whole plane lands on one line through the origin, or on the origin itself when \(\bm A = \bm 0\). Such a matrix is called singular.

The matrix \(\bm A = \begin{bmatrix} 2 & 1\\ 1 & 1/2\end{bmatrix}\) has \(\det\bm A = 2\cdot\tfrac12 - 1\cdot 1 = 0\) and its second column is half the first. The video below flattens the plane onto the line along \([2, 1]^\mathsf{T}\). The vectors \(\bm u = [1, 0.4]^\mathsf{T}\) and \(\bm v = [0.2, 2]^\mathsf{T}\) point in quite different directions, yet both land on the same point,

\[ \bm A\bm u = \begin{bmatrix} 2\cdot 1 + 1\cdot 0.4 \\ 1\cdot 1 + \tfrac12\cdot 0.4\end{bmatrix} = \begin{bmatrix} 2.4 \\ 1.2\end{bmatrix} = \bm A\bm v \]

From the image alone we cannot tell which of the two we started from. That loss is why a singular map has no inverse, as we see below.

Volume in space

A \(3\times 3\) matrix \(\bm A = [\bm a_1\ \bm a_2\ \bm a_3]\) carries the unit cube to the parallelepiped spanned by its columns, Figure 3.11.2. Its signed volume is the scalar triple product of Chapter 3.7, which is the determinant with the three vectors as columns,

\[ \det\bm A = \bm a_1\cdot(\bm a_2\times\bm a_3) \]

The plane reading carries over with volumes in place of areas. The map multiplies every volume by \(|\det\bm A|\). A negative determinant means that the columns form a left-handed system, and the map turns space inside out as a mirror does. A zero determinant means that the three columns lie in one plane, so that all of space collapses onto a plane, a line or a point.

Cofactor expansion

Beyond three dimensions there is no picture to read, and the determinant of an \(n\times n\) matrix is defined by reducing it, step by step, to determinants of order \(2\). We pick a row \(i\). For each entry \(a_{ij}\) of that row we strike out row \(i\) and column \(j\); the determinant of the \((n-1)\times(n-1)\) matrix that remains is the minor \(M_{ij}\), and with a sign attached it is the cofactor

\[ C_{ij} = (-1)^{i+j}M_{ij} \]

The determinant is the sum of the entries of the row times their cofactors,

\[ \boxed{\det\bm A = a_{i1}C_{i1} + a_{i2}C_{i2} + \dots + a_{in}C_{in}} \tag{3.11.2}\]

and expanding along a column \(j\) instead, \(\det\bm A = a_{1j}C_{1j} + \dots + a_{nj}C_{nj}\), gives the same value for every choice. For \(n > 3\) each minor is expanded in turn, until only determinants of order \(2\) remain. The factor \((-1)^{i+j}\) is \(+1\) or \(-1\) in a checkerboard pattern that starts with \(+\) in the top left corner,

\[ \begin{bmatrix} + & - & + & \cdots \\ - & + & - & \cdots \\ + & - & + & \cdots \\ \vdots & \vdots & \vdots & \ddots \end{bmatrix} \]

Each zero entry removes a whole term from 3.11.2, so we expand along the row or column with the most zeros. Expanding a \(3\times 3\) determinant along its first column gives back the triple product \(u_1(v_2w_3 - v_3w_2) + u_2(v_3w_1 - v_1w_3) + u_3(v_1w_2 - v_2w_1)\) of Chapter 3.7. Many texts write the determinant with vertical bars, \(|\bm A| = \det\bm A\), as we do for the minors below.

Rules for determinants

For square matrices \(\bm A\) and \(\bm B\) of order \(n\) and a number \(s\), the determinant obeys the rules

\[ \begin{aligned} \det\bm A^\mathsf{T} &= \det\bm A && \text{transpose} \\ \det(\bm A\bm B) &= \det\bm A\,\det\bm B && \text{product} \\ \det(s\bm A) &= s^n\det\bm A && \text{scalar multiple} \\ \det\bm I &= 1 && \text{identity} \end{aligned} \tag{3.11.3}\]

and for the rows of \(\bm A\):

1. multiplying one row by \(s\) multiplies the determinant by \(s\);
2. adding a multiple of one row to another leaves the determinant unchanged;
3. swapping two rows changes the sign of the determinant;
4. two equal rows give the determinant zero.

By the first rule of 3.11.3 the same four rules hold for the columns, and for columns we see them in the picture. A column multiplied by \(s\) stretches one side of the parallelogram by \(s\), and its area follows. Adding a multiple of \(\bm a_1\) to \(\bm a_2\) slides the tip of \(\bm a_2\) parallel to \(\bm a_1\), a shear that keeps the base and the height. Swapping the columns reverses the turn from the first to the second, and two equal columns span no area at all.

Multiplying the whole matrix by \(s\) stretches all \(n\) columns, so the volume grows by \(s^n\), not by \(s\). The determinant of a sum is in general no sum of determinants, \(\det(\bm A + \bm B)\neq\det\bm A + \det\bm B\).

The product rule says that areas scale twice. The product \(\bm A\bm B\) is the map \(\bm B\) followed by the map \(\bm A\) (Chapter 3.10): \(\bm B\) multiplies every area by \(\det\bm B\), then \(\bm A\) multiplies the result by \(\det\bm A\). In the video \(\bm B\) has \(\det\bm B = 1.5\) and \(\bm A\) has \(\det\bm A = 2\), and the readout of the current matrix ends at \(3\).

For \(2\times 2\) matrices SymPy confirms the product rule with general entries: the difference of the two sides expands to zero.

a11, a12, a21, a22, b11, b12, b21, b22 = sp.symbols('a11 a12 a21 a22 b11 b12 b21 b22', real=True)
AA = sp.Matrix([[a11, a12], [a21, a22]])
BB = sp.Matrix([[b11, b12], [b21, b22]])
diff = sp.expand((AA*BB).det() - AA.det()*BB.det())

A triangular matrix has zeros on one side of its diagonal. Expanding along its first column leaves one term, the first diagonal entry times the triangular minor below it, and repeating the step gives the product of the diagonal entries,

\[ \det\begin{bmatrix} u_{11} & u_{12} & \cdots & u_{1n} \\ 0 & u_{22} & \cdots & u_{2n} \\ \vdots & & \ddots & \vdots \\ 0 & 0 & \cdots & u_{nn}\end{bmatrix} = u_{11}u_{22}\cdots u_{nn} \tag{3.11.4}\]

The row rules turn any square matrix into a triangular one, and they keep track of the determinant on the way: rule 2 leaves it alone, and each swap of rule 3 flips its sign. This is the fast route of Figure 3.11.4.

The inverse: the map undone

The map \(\bm x\mapsto\bm A\bm x\) carries each vector to its image. The inverse map carries each image back to where it came from. Its matrix is the inverse \(\bm A^{-1}\), defined by

\[ \boxed{\bm A^{-1}\bm A = \bm A\bm A^{-1} = \bm I} \tag{3.11.5}\]

The first product undoes \(\bm A\) after it has acted, the second does \(\bm A\) after \(\bm A^{-1}\) has acted, and both leave every vector where it was. A matrix that has an inverse is called invertible. Only a square matrix can have one, since both products must be defined and equal to the same \(\bm I\). Played on the plane, the map \(\bm A = \begin{bmatrix} 2 & 1\\ 1 & 1\end{bmatrix}\) carries the plane and the vector \(\bm u\) away, and applying \(\bm A^{-1} = \begin{bmatrix} 1 & -1\\ -1 & 2\end{bmatrix}\) after it brings everything back to the identity.

Grant Sanderson shows the same picture in the video below and asks what happens when the map flattens the plane. Its second half, on column space and null space, returns in Chapter 3.12.

A matrix has at most one inverse. Suppose that \(\bm B\) and \(\bm C\) both satisfy 3.11.5. Inserting \(\bm I = \bm A\bm B\) and regrouping the product gives

\[ \bm C = \bm C\bm I = \bm C(\bm A\bm B) = (\bm C\bm A)\bm B = \bm I\bm B = \bm B \]

so the two are the same matrix, and we may speak of the inverse.

The determinant decides whether the inverse exists. Taking the determinant of 3.11.5 and using the product rule of 3.11.3 gives \(\det\bm A\,\det\bm A^{-1} = \det\bm I = 1\), so

\[ \det\bm A^{-1} = \frac{1}{\det\bm A} \]

which is impossible when \(\det\bm A = 0\). A singular matrix has no inverse, as the collapse of the plane told us: two vectors with the same image cannot both be sent back. The inverse undoes the area factor: a map that enlarges areas five times has an inverse that shrinks them to a fifth.

That \(\det\bm A\neq 0\) is also enough follows from a formula for the inverse, which we write out for \(n = 2\).

The \(2\times 2\) inverse by hand

For a \(2\times 2\) matrix with \(\det\bm A = a_{11}a_{22} - a_{12}a_{21}\neq 0\) the inverse is

\[ \boxed{\bm A^{-1} = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22}\end{bmatrix}^{-1} = \frac{1}{\det\bm A}\begin{bmatrix} a_{22} & -a_{12} \\ -a_{21} & a_{11}\end{bmatrix}} \tag{3.11.6}\]

We swap the two diagonal entries, change the sign of the other two, and divide by the determinant. Multiplying out confirms it, row times column,

\[ \begin{bmatrix} a_{22} & -a_{12} \\ -a_{21} & a_{11}\end{bmatrix}\begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22}\end{bmatrix} = \begin{bmatrix} a_{11}a_{22} - a_{12}a_{21} & a_{22}a_{12} - a_{12}a_{22} \\ a_{11}a_{21} - a_{21}a_{11} & a_{11}a_{22} - a_{12}a_{21}\end{bmatrix} = \det\bm A\;\bm I \]

and dividing by \(\det\bm A\) leaves \(\bm I\). The matrix in 3.11.6 holds the cofactors of \(\bm A\), transposed. That holds for every \(n\), \(\bm A^{-1} = \bm C^\mathsf{T}/\det\bm A\) with \(\bm C\) the matrix of cofactors \(C_{ij}\), the formula Cayley refers to in the quotation. With \(n\) cofactors of order \(n-1\) in every row, it inherits the cost of the expansion, so beyond \(n = 3\) we leave inverses to the computer.

Rules for inverses

For invertible matrices \(\bm A\) and \(\bm B\) of the same order and a number \(s\neq 0\),

\[ \begin{aligned} (\bm A^{-1})^{-1} &= \bm A \\ (\bm A^\mathsf{T})^{-1} &= (\bm A^{-1})^\mathsf{T} \\ (s\bm A)^{-1} &= \tfrac1s\,\bm A^{-1} \\ (\bm A\bm B)^{-1} &= \bm B^{-1}\bm A^{-1} \end{aligned} \tag{3.11.7}\]

Each rule is checked by multiplying the claimed inverse with the matrix and finding \(\bm I\). For the product,

\[ (\bm B^{-1}\bm A^{-1})(\bm A\bm B) = \bm B^{-1}(\bm A^{-1}\bm A)\bm B = \bm B^{-1}\bm I\bm B = \bm B^{-1}\bm B = \bm I \]

The order reverses. The map \(\bm A\bm B\) applies \(\bm B\) first and \(\bm A\) second, so undoing it removes \(\bm A\) first and \(\bm B\) second, as we take off the shoes before the socks. For the transpose, \((\bm A^{-1})^\mathsf{T}\bm A^\mathsf{T} = (\bm A\bm A^{-1})^\mathsf{T} = \bm I^\mathsf{T} = \bm I\) by the transpose rule of Chapter 3.10.

Orthogonal matrices

A square matrix \(\bm Q\) with

\[ \boxed{\bm Q^\mathsf{T}\bm Q = \bm I} \tag{3.11.8}\]

is called orthogonal. Its inverse costs nothing, \(\bm Q^{-1} = \bm Q^\mathsf{T}\). The entry \((i, j)\) of \(\bm Q^\mathsf{T}\bm Q\) is row \(i\) of \(\bm Q^\mathsf{T}\) times column \(j\) of \(\bm Q\), which is \(\bm q_i^\mathsf{T}\bm q_j\), the dot product of columns \(i\) and \(j\) (Chapter 3.6). The identity has ones on its diagonal and zeros elsewhere, so 3.11.8 says that every column has unit length and that any two columns are perpendicular: the columns of an orthogonal matrix are orthonormal.

An orthogonal map keeps dot products, and with them lengths and angles. For any vectors \(\bm a\) and \(\bm b\),

\[ (\bm Q\bm a)^\mathsf{T}(\bm Q\bm b) = \bm a^\mathsf{T}\bm Q^\mathsf{T}\bm Q\,\bm b = \bm a^\mathsf{T}\bm b \]

so \(|\bm Q\bm a| = |\bm a|\), and perpendicular vectors stay perpendicular. The determinant follows from 3.11.3, \(\det(\bm Q^\mathsf{T}\bm Q) = (\det\bm Q)^2 = \det\bm I = 1\), so \(\det\bm Q = \pm 1\): an orthogonal map keeps areas. With \(\det\bm Q = 1\) it is a rotation, with \(\det\bm Q = -1\) a reflection, which turns the plane over. The product of two orthogonal matrices is orthogonal, since \((\bm P\bm Q)^\mathsf{T}\bm P\bm Q = \bm Q^\mathsf{T}\bm P^\mathsf{T}\bm P\bm Q = \bm Q^\mathsf{T}\bm Q = \bm I\), and so are \(\bm Q^\mathsf{T}\) and \(\bm I\).