Matrices and vectors
A table of monthly sales has the commodities down the side and the outlets across the top. Which number is $a_{73}?$
Matrices are written as bold capitals, $\mathbf{A},$ $\mathbf{B}.$ A matrix with only one row is a row vector and one with only one column a column vector; both are written as bold lower-case letters, $\mathbf{a},$ $\mathbf{x},$ and their entries are called components. A vector with $n$ components is an $n$-vector.
If $m=n$ the matrix is square of order $n$, and then $a_{11},a_{22},\dots,a_{nn}$ form its main diagonal. Only a square matrix has one. (The order $m\times n$ is read "$m$ by $n$"; the dots $\dots$ and $\cdots$ stand for the entries that are not written out.)
So a single matrix equation between two $m\times n$ matrices is $mn$ ordinary equations at once. That compression is the whole reason for the notation.
A matrix from a formula
Write out $\mathbf{A}=(a_{ij})_{3\times 4}$ with $a_{ij}=2i-j.$
Try it yourself first. Then open one step at a time.
Row $i$ is obtained by letting $j$ run from $1$ to $4{:}$ $$ \mathbf{A}= \begin{pmatrix} 1 & 0 & -1 & -2\\ 3 & 2 & 1 & 0\\ 5 & 4 & 3 & 2 \end{pmatrix}\,. $$ For instance $a_{23}=2\cdot2-3=1.$
Each step to the right subtracts $1;$ each step down adds $2.$
$\mathbf{A}$ has the rows $(1,0,-1,-2),$ $(3,2,1,0)$ and $(5,4,3,2).$
What the indices mean
A chain has four outlets, each selling eight commodities. Let $a_{ij}$ be the value of sales of commodity $i$ at outlet $j$ in a given month. The data form an $8\times 4$ matrix $\mathbf{A}$: eight rows, one per commodity, and four columns, one per outlet. Then $a_{73}=225$ says that outlet $3$ sold $\text{\textdollar}225$ of commodity $7.$
Try it yourself first. Then open one step at a time.
Row $i$ of $\mathbf{A}$ is the sales of one commodity across all outlets; column $j$ is the whole product range of one outlet.
Which is which is decided entirely by the order of the two indices. Read the other way, $a_{73}$ would be commodity $3$ at outlet $7$ — and there is no outlet $7.$
A row is one commodity at every outlet, a column is one outlet's whole range; $a_{73}$ is commodity $7$ at outlet $3.$
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
What is $a_{23}$, and what is $a_{32}$?
- Why is that answer tempting? Mixes up rows and columns. $a_{32}=4$ reads the index column-first (column $3,$ row $2$): that entry is $a_{23}.$ And the $1$ sits in row $1,$ column $3,$ which is $a_{13}.$ This is the error the whole block is vulnerable to: a matrix with rows and columns swapped looks perfectly ordinary, and nothing in the notation warns you.
- Why is that answer tempting? Right about the first, and inventing the second. $\mathbf{A}$ is $2\times3,$ so there is no third row for $a_{32}$ to sit in: the index is not wrong, it is undefined.
- Why is that answer tempting? Counts from zero, as many programming languages do. Here the indices start at $1,$ and every formula of the lecture assumes it.
$a_{23}$ is row 2, column 3: the entry $4$.
$\mathbf{A}$ is $2 \times 3$ — two rows, three columns. So there is no row $3$, and $a_{32}$ is not a different entry, it is undefined.
Systems of linear equations
Two equations, three unknowns, and one unknown missing from the second equation. How do you write that as a block of numbers?
Nothing requires $m=n:$ there may be more equations than unknowns, or fewer.
The array of coefficients in (12.1) is an $m\times n$ matrix, the coefficient matrix $\mathbf{A}=(a_{ij})_{m\times n},$ and the right-hand sides form a column vector $\mathbf{b}.$ Note that the index convention of Definition 12.1 and the one in (12.1) agree: $i$ counts equations, which are rows, and $j$ counts variables, which are columns.
In words: a system of linear equations has constant terms on the right, and a solution must satisfy every equation at once. A system with a solution is consistent; one with none is inconsistent.
Recap Recap: linear equations
An equation is linear when each unknown appears only to the first power, multiplied by a constant, and never multiplied by another unknown. So $\sqrt3\,x_{1}+2x_{2}=5$ is linear, while equations containing $\sqrt{x_{1}},$ $x_{1}^{2}$ or $8x_{1}x_{2}$ are not.
With two unknowns a linear equation is a straight line, the graph of a linear function when it can be solved for $y.$ Two of them behave as Fact 1.7 says: exactly one solution (the lines cross), none (parallel and distinct) or infinitely many (the same line). Comparing slopes tells which. This lecture keeps the equations linear and lets their number grow.
Guess first. The question below is the next example. Answer it before you read on: what happens to an unknown that is simply absent from one of the equations?
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
What is the coefficient matrix $\mathbf{A}$?
- Why is that answer tempting? Treats the missing $x_{2}$ as a missing entry and leaves a ragged array. That is not a matrix at all: every row of an $m\times n$ matrix has exactly $n$ entries.
- Why is that answer tempting? The right numbers, with the equations as columns instead of rows. It is the index error of the first section, arriving in a more expensive place.
- Why is that answer tempting? Includes the right-hand sides. Those are $\mathbf{b},$ a separate vector, and the distinction matters: $\mathbf{A}$ multiplies the unknowns and $\mathbf{b}$ does not. (The array with $\mathbf{b}$ attached is a real object of the next lecture, but it is not the coefficient matrix.)
Two equations, three unknowns, so $\mathbf{A}$ is $2 \times 3$: the shape is decided before any number is written down.
$x_2$ is absent from the second equation, which means its coefficient there is $0$: $$\mathbf{A} = \begin{pmatrix} 3 & 2 & -1 \\ 1 & 0 & -5 \end{pmatrix}, \qquad \mathbf{b} = \begin{pmatrix} 4 \\ 7 \end{pmatrix}$$
A missing unknown is a zero
The system $$ \begin{aligned} 3x_{1} + 2x_{2} - x_{3} &= 4\\ x_{1} \phantom{{}+2x_{2}} \;-\; 5x_{3} &= 7 \end{aligned} $$ has two equations and three unknowns. What are its coefficient matrix and its right-hand side?
Try it yourself first. Then open one step at a time.
Two equations and three unknowns, so $$ \begin{aligned} \mathbf{A}&=\begin{pmatrix} 3 & 2 & -1\\ 1 & 0 & -5\end{pmatrix}\,,\\ \mathbf{b}&=\begin{pmatrix} 4\\ 7\end{pmatrix}\,. \end{aligned} $$
The missing $x_{2}$ in the second equation is a coefficient too: $a_{22}=0.$ A variable absent from an equation is a zero entry, not a gap.
$\mathbf{A}$ is $2\times3$ with rows $(3,2,-1)$ and $(1,0,-5);$ $\mathbf{b}=(4,7)$ as a column.
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
What is the coefficient matrix of the system $$ \begin{aligned} 2x_{1}-x_{3}&=5\,,\\ x_{2}+4x_{1}&=1\,? \end{aligned} $$
- Why is that answer tempting? Drops the missing unknowns and copies the coefficients in the order they are written. But column $j$ belongs to $x_{j}{:}$ the $-1$ multiplies $x_{3},$ not $x_{2}.$
- Why is that answer tempting? Three columns, but the coefficients are filled in as they appear on the page. The order in which the terms are written is irrelevant; each coefficient goes to the column of its unknown.
- Why is that answer tempting? The right numbers with the equations as columns: rows and columns swapped. Two equations give two rows.
Two equations in three unknowns: $\mathbf{A}$ is $2\times3.$ Row $1$ is the first equation, $2x_{1}+0x_{2}-1x_{3},$ and row $2$ the second, rewritten in the order of the unknowns as $4x_{1}+1x_{2}+0x_{3}.$ So $$ \mathbf{A}=\begin{pmatrix} 2 & 0 & -1\\ 4 & 1 & 0\end{pmatrix}\,. $$
Addition and scalar multiples
This month's sales and next month's sales are two tables of the same shape. What is the two months together, and what is the change?
Both operations act entry by entry, in the only way one could reasonably guess. ($\alpha$ is the Greek letter alpha; a scalar is just a number, and $\alpha\mathbf{A}$ multiplies every entry by it.)
In the chain of shops of Example 12.4: if $\mathbf{A}$ is this month's sales and $\mathbf{B}$ next month's, then $\mathbf{A}+\mathbf{B}$ is the two months together and $\mathbf{B}-\mathbf{A}$ the change, with a negative entry where sales fell.
Let $\mathbf{A},\mathbf{B},\mathbf{C}$ be $m\times n$ matrices, let $\alpha,\beta$ be scalars, and let $\mathbf{0}$ be the $m\times n$ zero matrix, all of whose entries are $0$. Then $$ \begin{array}{l} (\mathbf{A}+\mathbf{B})+\mathbf{C}=\mathbf{A}+(\mathbf{B}+\mathbf{C})\,, \\ \mathbf{A}+\mathbf{B}=\mathbf{B}+\mathbf{A}\,,\\[4pt] \mathbf{A}+\mathbf{0}=\mathbf{A}\,, \\ \mathbf{A}+(-\mathbf{A})=\mathbf{0}\,,\\[4pt] (\alpha+\beta)\mathbf{A}=\alpha\mathbf{A}+\beta\mathbf{A}\,, \\ \alpha(\mathbf{A}+\mathbf{B})=\alpha\mathbf{A}+\alpha\mathbf{B}\,. \end{array} $$
Every one of these holds entry by entry, where it is the corresponding rule for ordinary numbers. Associativity is what lets us write $\mathbf{A}+\mathbf{B}+\mathbf{C}$ without brackets. (The zero matrix plays the part of the number $0;$ $\beta$ is the Greek letter beta.)
Entry by entry
With $\mathbf{A}=\begin{pmatrix} 1 & 2\\ 0 & -1\end{pmatrix}$ and $\mathbf{B}=\begin{pmatrix} 4 & -2\\ 3 & 5\end{pmatrix},$ compute $\mathbf{A}+\mathbf{B},$ $3\mathbf{A}$ and $\mathbf{A}-\mathbf{B}.$
Try it yourself first. Then open one step at a time.
Both matrices are $2\times2,$ so all three exist. Adding entry by entry, $$ \mathbf{A}+\mathbf{B}=\begin{pmatrix} 5 & 0\\ 3 & 4\end{pmatrix}\,. $$
Multiplying every entry by $3,$ $$ 3\mathbf{A}=\begin{pmatrix} 3 & 6\\ 0 & -3\end{pmatrix}\,. $$
Subtracting entry by entry, $$ \mathbf{A}-\mathbf{B}=\begin{pmatrix} -3 & 4\\ -3 & -6\end{pmatrix}\,. $$
$\mathbf{A}+\mathbf{B}$ has rows $(5,0),(3,4);$ $3\mathbf{A}$ has rows $(3,6),(0,-3);$ $\mathbf{A}-\mathbf{B}$ has rows $(-3,4),(-3,-6).$
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
$\mathbf{A}$ is $2\times3,$ $\mathbf{B}$ is $3\times2,$ $\mathbf{r}=(1,2,3)$ is a row vector and $\mathbf{c}$ is the column vector with the same entries $1,2,3.$ Which of these is defined?
- Why is that answer tempting? Both have six entries, but the orders $2\times3$ and $3\times2$ differ. Addition pairs the entry in row $i,$ column $j$ of one with the entry in the same place of the other, and $\mathbf{A}$ has no row $3.$
- Why is that answer tempting? The same three numbers, but a $1\times3$ and a $3\times1$ matrix. Watch out 12.9 names exactly this case.
- Why is that answer tempting? Both have three rows, but $\mathbf{B}$ has two columns and $\mathbf{c}$ one. Same order means both numbers agree.
$3\mathbf{A}$ is $2\times3,$ like $\mathbf{A},$ so the sum exists, and by Fact 12.10 it is $(1+3)\mathbf{A}=4\mathbf{A}.$ The other three pairs differ in order: $$ \begin{aligned} &2\times3 \text{ and } 3\times2\,,\\ &1\times3 \text{ and } 3\times1\,,\\ &3\times2 \text{ and } 3\times1\,. \end{aligned} $$
Vectors and the inner product
You buy $5,$ $3,$ $6$ and $7$ kilos of four fruits at $4,$ $5,$ $3$ and $8$ dollars a kilo. Which operation on the two lists gives the bill?
A vector is a matrix, so it inherits everything above: $n$-vectors are added componentwise and scaled componentwise, and Fact 12.10 applies unchanged.
If $\mathbf{a}$ and $\mathbf{b}$ are commodity bundles, and $t$ people each buy $\mathbf{a}$ while $s$ people each buy $\mathbf{b},$ then $t\mathbf{a}+s\mathbf{b}$ is what the $t+s$ of them bought between them. One new operation is needed, and it is the one that turns quantities into money.
(So a linear combination is a bundle of bundles. For instance, with $\mathbf{a}=(1,2)$ and $\mathbf{b}=(3,0),$ $2\mathbf{a}+\mathbf{b}=(2+3,\,4+0)=(5,4).$)
The sign $\sum_{j=1}^{n}$ says: add the terms for $j=1,2,\dots,n.$ The dimension of a vector is its number of components. The inner product multiplies in pairs and adds.
Why it is the operation one wants
Buy the bundle $\mathbf{x}=(5,3,6,7)$ kilos of four fruits at prices $\mathbf{p}=(4,5,3,8)$ dollars per kilo. What is the bill?
Try it yourself first. Then open one step at a time.
Spending on the first fruit is $4\cdot5=20$ dollars, on the second $5\cdot3=15,$ and so on, so the total bill is $$ \begin{aligned} \mathbf{p}\cdot\mathbf{x} &= 4\cdot5+5\cdot3\\ &\quad+3\cdot6+8\cdot7\\ &= 109\,. \end{aligned} $$
The units confirm the shape of the definition: dollars per kilo times kilos is dollars, once per commodity, and then everything is added up.
$\mathbf{p}\cdot\mathbf{x}=109$ dollars: one number, the bill.
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
$\mathbf{p} = (4, 5, 3)$ is a price vector and $\mathbf{x} = (2, 0, 6)$ a quantity vector.
What is $\mathbf{p} \cdot \mathbf{x}$?
- Why is that answer tempting? That is the componentwise product, which is a perfectly good vector and answers a different question: spending on each good separately. The inner product adds them up.
- Why is that answer tempting? Concatenation, not multiplication. Nothing in the definition puts vectors end to end.
- Why is that answer tempting? A zero component is an ordinary number and multiplies fine. The only thing that makes an inner product undefined is a mismatch of dimensions.
A number: the total bill. Option A is the vector of what was spent on each good, and adding its components gives the same $26.$
For $n$-vectors $\mathbf{a},\mathbf{b},\mathbf{c}$ and a scalar $\alpha$:
- $\mathbf{a}\cdot\mathbf{b}=\mathbf{b}\cdot\mathbf{a}$;
- $(\alpha\mathbf{a})\cdot\mathbf{b} =\mathbf{a}\cdot(\alpha\mathbf{b}) =\alpha(\mathbf{a}\cdot\mathbf{b})$;
- $\mathbf{a}\cdot(\mathbf{b}+\mathbf{c}) =\mathbf{a}\cdot\mathbf{b}+\mathbf{a}\cdot\mathbf{c}$;
- $\mathbf{a}\cdot\mathbf{a}\ge 0$, with equality if and only if $\mathbf{a}=\mathbf{0}$.
Rules (a)–(c). (a)–(c) are the commutative and distributive laws for numbers, applied inside the sum.
Rule (d). For (d), $\mathbf{a}\cdot\mathbf{a}=a_{1}^{2}+\cdots+a_{n}^{2}$ is a sum of squares, so it is never negative, and it is zero exactly when every $a_{j}$ is zero.
(The rules (a)–(d) are the four lines of the Fact.) In prices and quantities, (b) says that doubling every price doubles the bill, and (c) that the bill for two bundles bought together is the sum of the two bills.
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
Two $n$-vectors have $\mathbf{a}\cdot\mathbf{b}=7.$ What is $(3\mathbf{a})\cdot(2\mathbf{b})?$
- Why is that answer tempting? Uses one of the two scalars and forgets the other. Rule (b) takes the $3$ out, and then the $2$ comes out the same way.
- Why is that answer tempting? $3\cdot7+2\cdot7,$ as if the scalars were added. Rule (b) pulls a scalar out as a factor; nothing in it adds.
- Why is that answer tempting? The rules of Fact 12.16 are exactly what lets us compute without them.
Rule (b) of Fact 12.16, used twice: $$ \begin{aligned} (3\mathbf{a})\cdot(2\mathbf{b})&=3\bigl(\mathbf{a}\cdot(2\mathbf{b})\bigr)\\ &=3\cdot2\,(\mathbf{a}\cdot\mathbf{b})\\ &=6\cdot7=42\,. \end{aligned} $$ In prices and quantities: three times the prices and twice the quantities make six times the bill.
Matrix multiplication
Each $y_{j}$ is a linear expression in $x_{1},x_{2},$ and each $z_{i}$ one in $y_{1},y_{2},y_{3}.$ What is each $z_{i}$ in terms of $x_{1},x_{2}?$
Addition was forced on us by the entries. Multiplication is not: the useful rule is not the entrywise one. To see where it comes from, substitute one linear system into another.
Suppose three variables $y_{1},y_{2},y_{3}$ are given in terms of $x_{1},x_{2},$ and two variables $z_{1},z_{2}$ are given in terms of the $y$'s: $$ \begin{aligned} y_{1} &= b_{11}x_{1}+b_{12}x_{2}\\ y_{2} &= b_{21}x_{1}+b_{22}x_{2}\\ y_{3} &= b_{31}x_{1}+b_{32}x_{2} \end{aligned} $$ $$ \begin{aligned} z_{1} &= a_{11}y_{1}+a_{12}y_{2}+a_{13}y_{3}\\ z_{2} &= a_{21}y_{1}+a_{22}y_{2}+a_{23}y_{3}\,. \end{aligned} $$ Then the $z$'s depend on the $x$'s. Substituting and collecting the terms in $x_{1},$ $$ \begin{aligned} z_{1} &= (a_{11}b_{11}+a_{12}b_{21}+a_{13}b_{31})\,x_{1}\\ &\quad + (\cdots)\,x_{2}\,, \end{aligned} $$ and the coefficient of $x_{1}$ in $z_{1}$ is exactly the inner product of the first row of $\mathbf{A}$ with the first column of $\mathbf{B}.$ The same happens in every slot. So if we want a product $\mathbf{A}\mathbf{B}$ whose entries are the coefficients of the composed system — and we do, because composing substitutions is what linear models constantly require — the definition is settled for us. The matrix product below is exactly that.
(Here $\mathbf{B}=(b_{jk})$ is the $3\times2$ matrix of the first system and $\mathbf{A}=(a_{ij})$ the $2\times3$ matrix of the second.)
A product, shape first
Let $$ \begin{aligned} \mathbf{A}&=\begin{pmatrix} 2 & -1 & 3\\ 0 & 4 & 1\end{pmatrix}\,,\\ \mathbf{B}&=\begin{pmatrix} 1 & 0\\ 2 & -3\\ 1 & 5\end{pmatrix}\,. \end{aligned} $$ Find $\mathbf{A}\mathbf{B},$ and the order of $\mathbf{B}\mathbf{A}.$
Try it yourself first. Then open one step at a time.
Shape first. $\mathbf{A}$ is $2\times3$ and $\mathbf{B}$ is $3\times2,$ so $\mathbf{A}\mathbf{B}$ is $2\times2.$
Four inner products. Its four entries are $$ \begin{aligned} c_{11} &= 2\cdot1+(-1)\cdot2+3\cdot1\\ &= 3\,,\\ c_{12} &= 2\cdot0+(-1)(-3)+3\cdot5\\ &= 18\,,\\ c_{21} &= 0\cdot1+4\cdot2+1\cdot1\\ &= 9\,,\\ c_{22} &= 0\cdot0+4(-3)+1\cdot5\\ &= -7\,, \end{aligned} $$ so $$ \mathbf{A}\mathbf{B}=\begin{pmatrix} 3 & 18\\ 9 & -7\end{pmatrix}\,. $$
The other order. Here $\mathbf{B}\mathbf{A}$ also exists, but it is $3\times3:$ not merely a different matrix, a different shape.
$\mathbf{A}\mathbf{B}$ has rows $(3,18)$ and $(9,-7);$ $\mathbf{B}\mathbf{A}$ is $3\times3.$
One row, one column, one number
Left $\mathbf{A},$ in the middle $\mathbf{B},$ right $\mathbf{A}\mathbf{B};$ under them the arithmetic for the entry being computed. The first setting is Example 12.20.
- Press Next entry four times. Each time one row of $\mathbf{A}$ meets one column of $\mathbf{B}$ and gives one number of $\mathbf{A}\mathbf{B}.$
- Press the shapes button once: the same $\mathbf{A}$ with a $2\times3$ matrix $\mathbf{B},$ $2\times3$ times $2\times3.$ Before you read the canvas, say why there is no product.
- Press it again for a $3\times2$ times $2\times4$ product. Predict its order, then press Fill it in and count the entries.
Guess first. Answer the next question without any arithmetic: write the two orders side by side for each product, and decide which pair has to agree and which pair is the answer.
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
$\mathbf{A}$ is $3 \times 4$ and $\mathbf{B}$ is $4 \times 3$.
Which of $\mathbf{AB}$ and $\mathbf{BA}$ exist, and what are their orders?
- Why is that answer tempting? Right about AB and too quick about BA. Check it the same way: B is 4 × 3 and A is 3 × 4, the inner two are both 3, so BA exists too.
- Why is that answer tempting? Right that both exist, and then assumes the answer cannot depend on the order. BA is 4 × 4 — the outer two numbers are B's rows and A's columns.
- Why is that answer tempting? Keeps the shape of A whatever it multiplies. The product takes its rows from the left factor and its columns from the right one, so it generally has the shape of neither.
Both exist, because the underlined inner numbers agree in each case. And the two products are not merely different matrices — they are different sizes.
In the practice hub the question "how many columns does $\mathbf{A}\mathbf{B}$ have?" has so far been answered correctly only about half the time (few answers yet), while the same question about rows does much better. The columns come from the right-hand factor: in $(m\times n)(n\times p)$ they are its $p.$
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
Let $$ \begin{aligned} \mathbf{A}&=\begin{pmatrix} 1 & 2 & 0\\ 3 & -1 & 4\end{pmatrix}\,,\\ \mathbf{B}&=\begin{pmatrix} 2 & 1\\ 1 & 5\\ -1 & 3\end{pmatrix}\,. \end{aligned} $$ What is the entry $c_{21}$ of $\mathbf{C}=\mathbf{A}\mathbf{B}?$
- Why is that answer tempting? That is $c_{12},$ row $1$ of $\mathbf{A}$ against column $2$ of $\mathbf{B}{:}$ the indices read column-first. The first index picks the row of the left factor.
- Why is that answer tempting? Row $2$ of $\mathbf{B}$ against column $1$ of $\mathbf{A},$ which is the entry of $\mathbf{B}\mathbf{A}.$ In $\mathbf{A}\mathbf{B}$ the row always comes from the left factor.
- Why is that answer tempting? The three products, not added. An entry of a matrix is a number: multiply in pairs and add.
$\mathbf{A}$ is $2\times3$ and $\mathbf{B}$ is $3\times2,$ so $\mathbf{C}$ is $2\times2.$ The entry $c_{21}$ is row $2$ of $\mathbf{A}$ against column $1$ of $\mathbf{B}{:}$ $$ \begin{aligned} c_{21}&=3\cdot2+(-1)\cdot1+4\cdot(-1)\\ &=6-1-4=1\,. \end{aligned} $$
A system is one equation
Two equations or two hundred: can a whole system be written as one equation, three characters long?
The definition pays for itself immediately.
Guess first. Before you read the Fact below, answer this: what is the product of a coefficient matrix and the column of unknowns?
Pick an answer before reading on. Getting it wrong is fine — that is how you find out what you believe.
$\mathbf{A}$ is the $m \times n$ coefficient matrix of a system, $\mathbf{x}$ the column of unknowns, $\mathbf{b}$ the column of right-hand sides.
What is $\mathbf{Ax}$?
- Why is that answer tempting? Confuses the matrix product with the inner product. One row of A against x gives a number; the whole of A against x gives one such number per row.
- Why is that answer tempting? Takes the shape from x rather than from A. Inner two agree (both n), outer two are the answer: m × 1, one entry per equation.
- Why is that answer tempting? A vector is a matrix with one column, so nothing new is needed. (m × n)(n × 1) is an ordinary product.
$(m \times n)(n \times 1) = m \times 1$, and the $i$th entry is row $i$ of $\mathbf{A}$ dotted with $\mathbf{x}$: $$a_{i1}x_1 + \cdots + a_{in}x_n$$ the left-hand side of equation $i$.
Two columns are equal exactly when all their entries agree. So $$\boxed{\;\mathbf{Ax} = \mathbf{b}\;}$$ is the system — all $m$ equations, in one line.
Let $\mathbf{A}$ be the coefficient matrix of the system (12.1), let $\mathbf{x}$ be the column vector of unknowns and $\mathbf{b}$ the column vector of constant terms. Then (12.1) says exactly $$ \mathbf{A}\mathbf{x}=\mathbf{b}\,. $$
The shape. $\mathbf{A}$ is $m\times n$ and $\mathbf{x}$ is $n\times1,$ so $\mathbf{A}\mathbf{x}$ is $m\times1$: a column of $m$ numbers, one per equation.
The entries. Its $i$th entry is the inner product of row $i$ of $\mathbf{A}$ with $\mathbf{x},$ which is $a_{i1}x_{1}+\cdots+a_{in}x_{n}$ — the left-hand side of the $i$th equation.
Equal columns. Two column vectors are equal exactly when all their entries agree, which is precisely the $m$ equations of (12.1).
An $m\times n$ system now looks like $ax=b$ from school, and the rest of the block is about how far that resemblance can be pushed: which rules of ordinary algebra survive (Lecture 13), and whether one can divide by $\mathbf{A}$ (Lecture 14).
Checking a solution with one product
This example is the app's. Write the system $$ \begin{aligned} x_{1}+2x_{2}&=5\\ 3x_{1}-x_{2}&=1\\ 2x_{1}+x_{2}&=4 \end{aligned} $$ as $\mathbf{A}\mathbf{x}=\mathbf{b},$ and decide with a matrix product whether $(1,2)$ and $(2,1)$ are solutions.
Try it yourself first. Then open one step at a time.
Shape first. Three equations in two unknowns, so $\mathbf{A}$ is $3\times2$ (Definition 12.5 does not need $m=n$): $$ \begin{aligned} \mathbf{A}&=\begin{pmatrix} 1 & 2\\ 3 & -1\\ 2 & 1\end{pmatrix}\,,\\ \mathbf{x}&=\begin{pmatrix} x_{1}\\ x_{2}\end{pmatrix}\,,\quad \mathbf{b}=\begin{pmatrix} 5\\ 1\\ 4\end{pmatrix}\,. \end{aligned} $$ $(3\times2)(2\times1)=3\times1,$ the order of $\mathbf{b}.$
The first candidate. Row by row, $$ \begin{aligned} &\begin{pmatrix} 1 & 2\\ 3 & -1\\ 2 & 1\end{pmatrix} \begin{pmatrix} 1\\ 2\end{pmatrix}\\ &=\begin{pmatrix} 1+4\\ 3-2\\ 2+2\end{pmatrix} =\begin{pmatrix} 5\\ 1\\ 4\end{pmatrix}\,, \end{aligned} $$ which is $\mathbf{b}$ entry by entry: $(1,2)$ solves all three equations.
The second candidate. The same two numbers in the other order: $$ \begin{aligned} &\begin{pmatrix} 1 & 2\\ 3 & -1\\ 2 & 1\end{pmatrix} \begin{pmatrix} 2\\ 1\end{pmatrix}\\ &=\begin{pmatrix} 4\\ 5\\ 5\end{pmatrix}\,. \end{aligned} $$ Already the first entry differs from $5,$ so $(2,1)$ is not a solution — Watch out 12.6 in action.
$\mathbf{A}$ has rows $(1,2),$ $(3,-1),$ $(2,1)$ and $\mathbf{b}=(5,1,4);$ $(1,2)$ is a solution and $(2,1)$ is not.
What you should take away
- An $m\times n$ matrix $\mathbf{A}=(a_{ij})$ has $m$ rows and $n$ columns, and $a_{ij}$ is the entry in row $i$, column $j$ — row index first, always. A matrix with one row or one column is a vector.
- Two matrices are equal when they have the same order and agree entry by entry, so one matrix equation is $mn$ ordinary ones.
- In a system of $m$ equations in $n$ unknowns, $a_{ij}$ is the coefficient of variable $x_{j}$ in equation $i$; a solution is an ordered list, and the system is consistent if one exists.
- Addition and scalar multiplication act entry by entry and need matrices of the same order. They obey the rules of ordinary algebra.
- The inner product $\mathbf{a}\cdot\mathbf{b}=\sum_{j}a_{j}b_{j}$ needs equal dimensions and returns a number. Prices $\cdot$ quantities $=$ money is the example to keep.
- $\mathbf{A}\mathbf{B}$ has $(i,k)$ entry equal to the inner product of row $i$ of $\mathbf{A}$ with column $k$ of $\mathbf{B}$. It exists only when the columns of $\mathbf{A}$ match the rows of $\mathbf{B}$, and then $(m\times n)(n\times p)=m\times p$.
- That rule is not arbitrary: it is exactly what makes substituting one linear system into another come out right. Its immediate reward is that any system becomes the single equation $\mathbf{A}\mathbf{x}=\mathbf{b}$.
Think further
Questions without computation. Think on your own or with a classmate before opening the thought.
Discussion question 1 — three operations, three verdicts
$\mathbf{A}$ is $3\times5$ and $\mathbf{B}$ is $5\times3$. Which of $\mathbf{A}+\mathbf{B}$, $\mathbf{A}\mathbf{B}$, $\mathbf{B}\mathbf{A}$ are defined, and what are their orders?
One way to think about it
$\mathbf{A}+\mathbf{B}$ is not defined: the orders $3\times5$ and $5\times3$ differ (Watch out 12.9), although both have fifteen entries. Both products exist, because in each the inner two numbers agree: $$ \begin{aligned} (3\times5)(5\times3)&=3\times3\,,\\ (5\times3)(3\times5)&=5\times5\,. \end{aligned} $$ So $\mathbf{A}\mathbf{B}$ is $3\times3$ and $\mathbf{B}\mathbf{A}$ is $5\times5:$ the same two matrices, and products of different sizes.
Discussion question 2 — can a·a be negative?
$\mathbf{a}=(1,-2,3)$ and $\mathbf{b}=(-3,2,5)$. Compute $\mathbf{a}\cdot\mathbf{b}$, and then $\mathbf{a}\cdot\mathbf{a}$. Can the second ever be negative?
One way to think about it
$$ \begin{aligned} \mathbf{a}\cdot\mathbf{b}&=-3-4+15=8\,,\\ \mathbf{a}\cdot\mathbf{a}&=1+4+9=14\,. \end{aligned} $$ The second can never be negative. $\mathbf{a}\cdot\mathbf{a}$ is the sum of the squares of the components, and a square is never negative; it is $0$ only for the zero vector. That is rule (d) of Fact 12.16. An inner product of two different vectors can be negative, of course: $(1,0)\cdot(-1,0)=-1.$
Discussion question 3 — a system multiplied out
Write the system $2x_{1}-x_{2}=5$, $x_{1}+3x_{2}=1$ in the form $\mathbf{A}\mathbf{x}=\mathbf{b}$, and check by multiplying out that you recover the two equations.
One way to think about it
$$ \begin{aligned} \mathbf{A}&=\begin{pmatrix} 2 & -1\\ 1 & 3\end{pmatrix}\,,\\ \mathbf{x}&=\begin{pmatrix} x_{1}\\ x_{2}\end{pmatrix}\,,\quad \mathbf{b}=\begin{pmatrix} 5\\ 1\end{pmatrix}\,. \end{aligned} $$ Multiplying out, row by row, $$ \mathbf{A}\mathbf{x}=\begin{pmatrix} 2x_{1}-x_{2}\\ x_{1}+3x_{2}\end{pmatrix}\,, $$ and this column equals $\mathbf{b}$ exactly when $2x_{1}-x_{2}=5$ and $x_{1}+3x_{2}=1$ (Definition 12.2): the two equations, and nothing else (Fact 12.21).
Discussion question 4 — must AB equal BA?
$\mathbf{A}$ is $4\times4$ and $\mathbf{B}$ is $4\times4$. Must $\mathbf{A}\mathbf{B}=\mathbf{B}\mathbf{A}$?
(A) yes, both products exist and have the same order;
(B) yes, because multiplication of numbers commutes;
(C) no — both exist and have the same order, and that is no reason for them to be equal;
(D) only if $\mathbf{A}$ and $\mathbf{B}$ are diagonal.
One way to think about it
(C). (A) confuses existing with being equal: the orders decide only whether a product exists. (B) forgets what an entry of a product is: the entry $(i,k)$ of $\mathbf{A}\mathbf{B}$ uses row $i$ of $\mathbf{A}$ and column $k$ of $\mathbf{B},$ and of $\mathbf{B}\mathbf{A}$ row $i$ of $\mathbf{B}$ and column $k$ of $\mathbf{A}$ — different numbers, so the commutativity of numbers does not help. (D) is too strong: every matrix commutes with itself, $\mathbf{A}\mathbf{A}=\mathbf{A}\mathbf{A},$ diagonal or not.
An example (the app's), in the $2\times2$ case: $$ \begin{aligned} \begin{pmatrix} 1 & 1\\ 0 & 1\end{pmatrix} \begin{pmatrix} 1 & 0\\ 1 & 1\end{pmatrix} &=\begin{pmatrix} 2 & 1\\ 1 & 1\end{pmatrix}\,,\\ \begin{pmatrix} 1 & 0\\ 1 & 1\end{pmatrix} \begin{pmatrix} 1 & 1\\ 0 & 1\end{pmatrix} &=\begin{pmatrix} 1 & 1\\ 1 & 2\end{pmatrix}\,. \end{aligned} $$ Put these two matrices in the top left corner of two $4\times4$ matrices, with zeros everywhere else, and the two products differ in the same corner.