3.10  Matrices as maps

Entries, rows and columns

Two matrices are equal, \(\bm A = \bm B\), when they have the same size and equal entries in every place, \(a_{ij} = b_{ij}\) for all \(i\) and \(j\). A matrix whose entries are all zero is a zero matrix, written \(\bm 0\). There is one zero matrix of each size, and two zero matrices of different sizes are different matrices.

Sum and scalar multiple

Two matrices of the same size are added entry by entry, and a number \(s\) multiplies every entry,

\[ \bm A + \bm B = [a_{ij} + b_{ij}], \qquad s\bm A = [s\,a_{ij}] \tag{3.10.2}\]

The difference is \(\bm A - \bm B = \bm A + (-1)\bm B\). Matrices of different sizes cannot be added. Because each entry follows the arithmetic of numbers, the sum and the scalar multiple obey the rules

\[ \begin{aligned} \bm A + \bm B &= \bm B + \bm A && \text{commutative law} \\ (\bm A + \bm B) + \bm C &= \bm A + (\bm B + \bm C) && \text{associative law} \\ \bm A + \bm 0 &= \bm A && \text{zero matrix} \\ s(\bm A + \bm B) &= s\bm A + s\bm B, \quad (s + t)\bm A = s\bm A + t\bm A && \text{distributive laws} \\ s(t\bm A) &= (st)\bm A && \text{associative law for scalars} \end{aligned} \]

for matrices \(\bm A\), \(\bm B\), \(\bm C\) of one size and numbers \(s\) and \(t\). These are the rules of vectors from Chapter 3.5, now for tables of any shape.

A matrix acts on a vector

Every vector of the plane is a combination of the basis vectors, \(\bm x = x_1\bm i + x_2\bm j\). Consider a map \(T\) that sends each vector \(\bm x\) to a new vector \(T(\bm x)\) and that is linear: it keeps straight lines straight, keeps evenly spaced points on a line evenly spaced, and leaves the origin where it is. For vectors these requirements read

\[ T(\bm u + \bm v) = T(\bm u) + T(\bm v), \qquad T(s\bm u) = s\,T(\bm u) \]

and applied to \(\bm x\) they give

\[ T(\bm x) = x_1\,T(\bm i) + x_2\,T(\bm j) \]

A linear map is known as soon as we know where it sends \(\bm i\) and \(\bm j\). We write the two images as the columns of a matrix, \(\bm a_1 = T(\bm i)\) and \(\bm a_2 = T(\bm j)\), so that \(\bm A = [\bm a_1\ \bm a_2]\), and define the product of the matrix and a vector as this combination of its columns,

\[ \boxed{\bm A\bm x = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = x_1\begin{bmatrix} a_{11} \\ a_{21} \end{bmatrix} + x_2\begin{bmatrix} a_{12} \\ a_{22} \end{bmatrix} = x_1\bm a_1 + x_2\bm a_2} \tag{3.10.3}\]

With \(\bm x = \bm i = [1, 0]^\mathsf{T}\) the combination is \(\bm a_1\), and with \(\bm x = \bm j\) it is \(\bm a_2\):

Adding the two columns of 3.10.3 row by row gives the same vector in a second form,

\[ \bm A\bm x = \begin{bmatrix} a_{11}x_1 + a_{12}x_2 \\ a_{21}x_1 + a_{22}x_2 \end{bmatrix} \tag{3.10.4}\]

where each component is the dot product of a row of \(\bm A\) with \(\bm x\) (Chapter 3.6). The column form says what the map does, and the row form is the quicker one by hand. Nothing in the argument depends on the plane. An \(m\times n\) matrix has \(n\) columns with \(m\) components each, so \(\bm A\bm x = x_1\bm a_1 + \dots + x_n\bm a_n\) is defined when \(\bm x\) has \(n\) components, and the result has \(m\).

The video plays six maps of the plane. The faint fixed grid marks where the plane was and the blue grid where it goes. The matrix in the corner is read off the arrows \(\bm i\) and \(\bm j\) as they move, its columns being their tips, and \(\bm u\) and \(\bm v\) are carried along as \(\bm A\bm u\) and \(\bm A\bm v\). At every moment the grid lines stay straight and evenly spaced. The green unit square turns orange when a map turns the plane over, which Chapter 3.11 measures with the determinant.

Each matrix in Table 3.10.1 is read off the picture. The shear keeps \(\bm i\) and pushes \(\bm j\) sideways to \([k, 1]^\mathsf{T}\), and the projection sends \(\bm j\) to the zero vector, so its second column is zero and the whole plane falls onto the \(x\) axis.

Grant Sanderson tells the same story with other maps of the plane and shows how to read any \(2\times 2\) matrix as one.

Two maps in sequence: the matrix product

Let \(\bm B\) act first and \(\bm A\) second. A vector \(\bm x\) goes to \(\bm B\bm x\) and then to \(\bm A(\bm B\bm x)\). Two linear maps in sequence form a linear map, and by 3.10.3 its matrix has as columns the images of the basis vectors. The first one is \(\bm A(\bm B\bm i) = \bm A\bm b_1\), where \(\bm b_1\) is the first column of \(\bm B\), and so on. We call that matrix the product \(\bm A\bm B\),

\[ \boxed{\bm A\bm B = \bm A\,[\bm b_1\ \bm b_2\ \cdots\ \bm b_n] = [\bm A\bm b_1\ \bm A\bm b_2\ \cdots\ \bm A\bm b_n]} \tag{3.10.5}\]

so that \((\bm A\bm B)\bm x = \bm A(\bm B\bm x)\) for every \(\bm x\). A product of maps is read from right to left: the factor next to \(\bm x\) acts first.

Entry \(c_{ij}\) of \(\bm C = \bm A\bm B\) sits in column \(j\), which is \(\bm A\bm b_j\), and by 3.10.4 its component \(i\) is the dot product of row \(i\) of \(\bm A\) with column \(j\) of \(\bm B\),

\[ c_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \dots + a_{ip}b_{pj} = \sum_{k=1}^{p} a_{ik}b_{kj} \tag{3.10.6}\]

the rule of row times column. A row of \(\bm A\) and a column of \(\bm B\) can only be paired when they have equally many entries, so \(\bm A\bm B\) is defined only when \(\bm A\) has as many columns as \(\bm B\) has rows. The result has the rows of \(\bm A\) and the columns of \(\bm B\):

Grant Sanderson builds the product from two maps in sequence and computes it column by column.

Rules of the product

For matrices whose sizes allow the products and for a number \(s\),

\[ \begin{aligned} (\bm A\bm B)\bm C &= \bm A(\bm B\bm C) && \text{associative law} \\ \bm A(\bm B + \bm C) &= \bm A\bm B + \bm A\bm C && \text{distributive law from the left} \\ (\bm A + \bm B)\bm C &= \bm A\bm C + \bm B\bm C && \text{distributive law from the right} \\ s(\bm A\bm B) &= (s\bm A)\bm B = \bm A(s\bm B) && \text{scalar factor} \end{aligned} \tag{3.10.7}\]

The associative law needs no computation in the picture of maps: both sides apply \(\bm C\), then \(\bm B\), then \(\bm A\). So brackets in a product may be dropped, \(\bm A\bm B\bm C\), but the order of the factors may not be changed, since in general

When the two products happen to be equal, \(\bm A\) and \(\bm B\) are said to commute. A second habit from numbers also fails: \(\bm A\bm B = \bm 0\) does not need a zero factor. Projecting on the \(x\) axis and then on the \(y\) axis sends every vector of the plane to \(\bm 0\), although neither projection is the zero matrix.

Transpose

The transpose \(\bm A^\mathsf{T}\) of an \(m\times n\) matrix \(\bm A\) is the \(n\times m\) matrix whose rows are the columns of \(\bm A\), entry by entry \((\bm A^\mathsf{T})_{ij} = a_{ji}\). For instance

\[ \begin{bmatrix} 2 & 5 & 4 \\ -1 & 3 & 0 \end{bmatrix}^\mathsf{T} = \begin{bmatrix} 2 & -1 \\ 5 & 3 \\ 4 & 0 \end{bmatrix} \]

The transpose obeys

\[ \begin{aligned} (\bm A^\mathsf{T})^\mathsf{T} &= \bm A, \qquad (\bm A + \bm B)^\mathsf{T} = \bm A^\mathsf{T} + \bm B^\mathsf{T}, \qquad (s\bm A)^\mathsf{T} = s\bm A^\mathsf{T} \\ (\bm A\bm B)^\mathsf{T} &= \bm B^\mathsf{T}\bm A^\mathsf{T} \end{aligned} \tag{3.10.8}\]

and the last rule reverses the order. Entry \(ij\) of \((\bm A\bm B)^\mathsf{T}\) is entry \(ji\) of \(\bm A\bm B\), and by 3.10.6

\[ \big((\bm A\bm B)^\mathsf{T}\big)_{ij} = \sum_k a_{jk}b_{ki} = \sum_k (\bm B^\mathsf{T})_{ik}(\bm A^\mathsf{T})_{kj} = (\bm B^\mathsf{T}\bm A^\mathsf{T})_{ij} \]

The sizes demand the reversal as well: with \(\bm A\) of size \(m\times p\) and \(\bm B\) of size \(p\times n\), the product \(\bm B^\mathsf{T}\bm A^\mathsf{T}\) is \((n\times p)(p\times m)\), while \(\bm A^\mathsf{T}\bm B^\mathsf{T}\) is in general not defined.

A column vector is an \(n\times 1\) matrix, and its transpose is a row. The product of the row \(\bm a^\mathsf{T}\) and the column \(\bm b\) is a \(1\times 1\) matrix, row times column,

\[ \bm a^\mathsf{T}\bm b = \begin{bmatrix} a_1 & a_2 & a_3 \end{bmatrix}\begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix} = a_1b_1 + a_2b_2 + a_3b_3 = \bm a\cdot\bm b \tag{3.10.9}\]

the dot product of Chapter 3.6 written as a matrix product. In SymPy the transpose is .T. With the matrices of Example 5 and with \(\bm a = [2, 1, 3]^\mathsf{T}\) and \(\bm b = [1, -1, 2]^\mathsf{T}\) we check the product rule and 3.10.9:

AA = sp.Matrix([[6, 4, -1], [-3, 5, 2]])
BB = sp.Matrix([[7, 4], [5, -8], [3, 6]])
rule = (AA*BB).T - BB.T*AA.T          # the zero matrix if the rule holds

aa = sp.Matrix([2, 1, 3])
bb = sp.Matrix([1, -1, 2])
aTb = aa.T*bb                         # a 1x1 matrix

which gives

The product aa.T*bb is the \(1\times 1\) matrix \([7]\), and aTb[0] takes out the number.

Square, triangular and symmetric matrices

An \(n\times n\) matrix is square. Its entries \(a_{11}, a_{22}, \dots, a_{nn}\) form the main diagonal. A square matrix with only zeros below the diagonal is upper triangular, one with only zeros above it lower triangular,

\[ \begin{bmatrix} 2 & 5 & 9 \\ 0 & 7 & 1 \\ 0 & 0 & 3 \end{bmatrix}, \qquad \begin{bmatrix} 7 & 0 & 0 & 0 \\ 1 & 5 & 0 & 0 \\ 8 & 9 & 4 & 0 \\ -1 & 3 & 5 & 3 \end{bmatrix} \]

A square matrix is symmetric when \(\bm A^\mathsf{T} = \bm A\), that is \(a_{ji} = a_{ij}\): the diagonal is a mirror for the entries. It is antisymmetric when \(\bm A^\mathsf{T} = -\bm A\). On the diagonal that reads \(a_{ii} = -a_{ii}\), so an antisymmetric matrix has zeros there. Every square matrix splits into a symmetric part and an antisymmetric part,

\[ \bm A = \underbrace{\tfrac12(\bm A + \bm A^\mathsf{T})}_{\bm S} + \underbrace{\tfrac12(\bm A - \bm A^\mathsf{T})}_{\bm K} \tag{3.10.10}\]

and the rules 3.10.8 confirm the two parts, \(\bm S^\mathsf{T} = \tfrac12(\bm A^\mathsf{T} + \bm A) = \bm S\) and \(\bm K^\mathsf{T} = \tfrac12(\bm A^\mathsf{T} - \bm A) = -\bm K\).

Diagonal matrices and the identity

A square matrix whose entries off the diagonal are all zero is a diagonal matrix, written \(\bm D = \operatorname{diag}(d_1, d_2, \dots, d_n)\). As a map it stretches each axis by its own factor, the stretch of Table 3.10.1. The diagonal matrix with ones on the diagonal is the identity matrix \(\bm I\). Its columns are the basis vectors themselves, so it leaves every vector where it is, and for every \(\bm A\) of a fitting size

\(\bm I\) plays the part of the number 1. There is one identity matrix of each size, and \(\bm I\bm A\) and \(\bm A\bm I\) use different ones when \(\bm A\) is not square.

A diagonal matrix multiplies in a simple pattern. By 3.10.6 the sum in \((\bm D\bm A)_{ij}\) keeps only the term \(k = i\), and the sum in \((\bm A\bm D)_{ij}\) only the term \(k = j\),

\[ (\bm D\bm A)_{ij} = d_i\,a_{ij}, \qquad (\bm A\bm D)_{ij} = a_{ij}\,d_j \tag{3.10.11}\]

From the left, \(\bm D\) multiplies row \(i\) of \(\bm A\) by \(d_i\). From the right, it multiplies column \(j\) by \(d_j\).

Powers

A square matrix can be multiplied by itself, and its powers are defined as for numbers,

\[ \bm A^0 = \bm I, \qquad \bm A^n = \bm A\,\bm A^{n-1}, \quad n = 1, 2, \dots \]

so that \(\bm A^2 = \bm A\bm A\) and \(\bm A^3 = \bm A\bm A\bm A\). As a map, \(\bm A^n\) applies \(\bm A\) \(n\) times. The laws \(\bm A^m\bm A^n = \bm A^{m + n}\) and \((\bm A^m)^n = \bm A^{mn}\) hold, since all the factors are the same matrix and the order cannot matter.

Matrix-valued functions

When the entries of a matrix are functions of a variable \(x\), the matrix is a function of \(x\) too, \(\bm A(x) = [a_{ij}(x)]\), as a vector function was a function of its parameter in Chapter 3.9. Its derivative and its integral are taken entry by entry,

\[ \frac{\mathrm d\bm A}{\mathrm dx} = \left[\frac{\mathrm da_{ij}}{\mathrm dx}\right], \qquad \int\bm A\,\mathrm dx = \left[\int a_{ij}\,\mathrm dx\right] \tag{3.10.12}\]

and the rules of the derivative carry over, with one care: in the product rule

\[ \frac{\mathrm d}{\mathrm dx}(\bm A\bm B) = \frac{\mathrm d\bm A}{\mathrm dx}\bm B + \bm A\frac{\mathrm d\bm B}{\mathrm dx} \]

each term keeps the factors in their order.

Maps in space and between dimensions

Everything above holds in space. A \(3\times 3\) matrix has three columns, the images of \(\bm i\), \(\bm j\) and \(\bm k\), and by 3.10.3 it sends \(\bm x = x_1\bm i + x_2\bm j + x_3\bm k\) to \(x_1\bm a_1 + x_2\bm a_2 + x_3\bm a_3\). In the video the unit cube is carried by

\[ \bm A = \begin{bmatrix} 1 & 1 & 0 \\ 0 & 2 & 0 \\ 1 & 0 & 1 \end{bmatrix} \]

Its edges stay straight and parallel, and its three edges from the origin end on the three columns of \(\bm A\); \(\bm k\) stays where it was, since the third column is \(\bm k\).

A matrix need not be square. A \(3\times 2\) matrix has two columns with three components each: it sends the two basis vectors of the plane to two vectors in space, and the whole plane to the plane through the origin that those two columns span. A \(2\times 3\) matrix goes the other way, from space to the plane. The simplest one drops the \(z\) component,

\[ \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \]

the view of space from above. The size rule \((m\times n)(n\times 1) = m\times 1\) states the same: the matrix takes vectors with as many components as it has columns and returns vectors with as many components as it has rows.