Interactive guide · linear algebra

Operations with matrices

How matrices are added and multiplied: sum, product by a number, matrix product (row times column), block product and the trace. Click any card to see the step-by-step proof.

Buy me a coffee
Your matrices
A =
Rows Columns
B =
Rows Columns
Examples
Operation
Result
A · 2 × 2
1234
·
B · 2 × 2
0110
=
C · 2 × 2

(2 × 2) · (2 × 2) gives a (2 × 2) matrix: the columns of the first and the rows of the second match and disappear.

One entry, step by step
c₁₁ = a₁₁·b₁₁ + a₁₂·b₂₁
= 1·0 + 2·1
= 2

Press another entry of the result. In a product, the row of the first matrix and the column of the second are highlighted; they are multiplied in pairs and added up.

Does the order matter?

AB ≠ BA: for example, at c₁₁ one gives 2 and the other 3.

With numbers, 2 · 3 = 3 · 2. With matrices not always: the product is not commutative.

01 · Sum

Sum of matrices

Two matrices are added by adding the entries in the same position. That is why only matrices of the same size can be added.

SumIf A = [aij]n×m and B = [bij]n×m, then A + B = [aij + bij]n×m.
  1. 1Worked example. Both matrices are 2 × 3, so they can be added.(1203−14)+(205−112)\begin{pmatrix} 1 & 2 & 0 \\ 3 & -1 & 4 \end{pmatrix} + \begin{pmatrix} 2 & 0 & 5 \\ -1 & 1 & 2 \end{pmatrix}
  2. 2Add each entry to the one in the same position: 1 + 2, 2 + 0, 0 + 5 in the first row and 3 + (−1), (−1) + 1, 4 + 2 in the second.(1+22+00+53−1−1+14+2)=(325206)\begin{pmatrix} 1 + 2 & 2 + 0 & 0 + 5 \\ 3 - 1 & -1 + 1 & 4 + 2 \end{pmatrix} = \begin{pmatrix} 3 & 2 & 5 \\ 2 & 0 & 6 \end{pmatrix}

What it is for. Adding matrices is adding tables of data position by position: the sales of two shops, two images (grids of pixels) or two linear transformations at once.

02 · Scalar product

Product by a scalar

A scalar is a number (here λ ∈ K). Multiplying a matrix by a scalar means multiplying all its entries by that number.

Product by a scalarλA = [λaij]n×m. The size does not change.
  1. 1Worked example: multiply the matrix A by 3.3(1−204)3 \begin{pmatrix} 1 & -2 \\ 0 & 4 \end{pmatrix}
  2. 2Each entry is multiplied by 3.(3⋅13⋅(−2)3⋅03⋅4)=(3−6012)\begin{pmatrix} 3 \cdot 1 & 3 \cdot (-2) \\ 3 \cdot 0 & 3 \cdot 4 \end{pmatrix} = \begin{pmatrix} 3 & -6 \\ 0 & 12 \end{pmatrix}

What it is for. It is used to change scale (raise every price by 10 %, brighten an image) and, together with the sum, to combine matrices: 2A − 3B is a combination of two matrices.

03 · Properties

Properties of the sum and of the product by a scalar

Let λ, μ ∈ K be numbers and A, B, C matrices of the same size n × m. Since everything is done entry by entry, the same rules as for numbers hold.

Associative(A + B) + C = A + (B + C)See why
CommutativeA + B = B + ASee why
Zero elementA + 0 = 0 + A = A, where 0 is the matrix with every entry 0.See why
OppositeThere is a B with A + B = B + A = 0: it is the opposite, −A (the same entries with the sign changed).See why
Distributive over the sum of matricesλ(A + B) = λA + λBSee why
Distributive over the sum of scalars(λ + μ)A = λA + μASee why
Mixed associative(λμ)A = λ(μA)See why

What it is for. They guarantee that you can reorder, group and solve with matrices just as with numbers, as long as you only add and multiply by scalars. For example, from A + X = B you get X = B − A.

04 · Matrix product

Multiplying matrices: row times column

Multiplying matrices is trickier than adding them. The basic piece is multiplying a row by a column with the same number of entries.

Row times columnMultiply the entries in pairs and add: (a1 a2 … ap) · (b1, b2, …, bp)t = a1b1 + a2b2 + … + apbp. The result is a number.See why
ConditionTo be able to multiply AB, the number of columns of A must equal the number of rows of B.See why
SizeA = [aij]n×p, B = [bij]p×m give C = AB of size n × m: the rows of A and the columns of B.
Each entry of Ccij is row i of A times column j of B, that is, cij = Σk=1p aikbkj. (The sum index is k, so it is not confused with the size p.)See why
  1. 1Worked example. A is 2 × 3 and B is 3 × 2: the columns of A (3) match the rows of B (3), so they can be multiplied and the result is 2 × 2.(123456)(789101112)\begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{pmatrix} \begin{pmatrix} 7 & 8 \\ 9 & 10 \\ 11 & 12 \end{pmatrix}
  2. 2c₁₁: row 1 of A times column 1 of B.c11=1⋅7+2⋅9+3⋅11=7+18+33=58c_{11} = 1 \cdot 7 + 2 \cdot 9 + 3 \cdot 11 = 7 + 18 + 33 = 58
  3. 3c₁₂: row 1 of A times column 2 of B.c12=1⋅8+2⋅10+3⋅12=8+20+36=64c_{12} = 1 \cdot 8 + 2 \cdot 10 + 3 \cdot 12 = 8 + 20 + 36 = 64
  4. 4c₂₁: row 2 of A times column 1 of B.c21=4⋅7+5⋅9+6⋅11=28+45+66=139c_{21} = 4 \cdot 7 + 5 \cdot 9 + 6 \cdot 11 = 28 + 45 + 66 = 139
  5. 5c₂₂: row 2 of A times column 2 of B.c22=4⋅8+5⋅10+6⋅12=32+50+72=154c_{22} = 4 \cdot 8 + 5 \cdot 10 + 6 \cdot 12 = 32 + 50 + 72 = 154
  6. 6Everything is put together in the product matrix.AB=(5864139154)AB = \begin{pmatrix} 58 & 64 \\ 139 & 154 \end{pmatrix}

Size of the product: (n × p)(p × m) = n × m

(n×p)⋅(p×m)=(n×m)(n \times p)\cdot(p \times m) = (n \times m)

To multiply, the columns of A must match the rows of B; the result takes the rows of A and the columns of B.

See proof

The product of matrices: row times column

cij=∑k=1paik bkjc_{ij} = \sum_{k=1}^{p} a_{ik}\, b_{kj}

Each entry of AB is a row of A times a column of B: a sum of products.

See proof

What it is for. The matrix product composes transformations (apply one and then another), computes totals (prices times quantities) and lets you write a system of equations as AX = b. That is why it is defined in such a particular way.

05 · Product properties

Properties of the matrix product

The product satisfies almost all the rules for numbers, with one very important exception: the order of the factors does matter.

Associative property: (AB)C = A(BC)

(AB)C=A(BC)(AB)C = A(BC)

You can group the products however you like; the result is the same.

See proof

Distributive property: (A + B)C = AC + BC

(A+B)C=AC+BC,C(A+B)=CA+CB(A + B)C = AC + BC,\qquad C(A + B) = CA + CB

The product distributes over the sum, both on the right and on the left.

See proof

The identity matrix is the neutral element: IA = AI = A

IA=AI=AIA = AI = A

Multiplying by the identity (on either side) leaves the matrix unchanged.

See proof

The product is not commutative: AB ≠ BA

AB≠BAAB \neq BA

In general, the order of the factors does change the product.

See proof
Mind the orderIn general AB ≠ BA. So you cannot "simplify" or swap the order of factors: (A + B)² is not A² + 2AB + B², but A² + AB + BA + B².

What it is for. With these rules you expand and factor expressions, and change the grouping to save computations. Knowing what does not hold (commutativity) avoids the most typical mistakes.

06 · By blocks

Block multiplication

A big matrix can be split into rectangular pieces called blocks. If the partitions are compatible (the columns of A are cut in the same way as the rows of B), the blocks multiply as if they were numbers.

Block multiplication

(A11A12A21A22)(B11B12B21B22)=(A11B11+A12B21A11B12+A12B22A21B11+A22B21A21B12+A22B22)\begin{pmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{pmatrix} \begin{pmatrix} B_{11} & B_{12} \\ B_{21} & B_{22} \end{pmatrix} = \begin{pmatrix} A_{11}B_{11} + A_{12}B_{21} & A_{11}B_{12} + A_{12}B_{22} \\ A_{21}B_{11} + A_{22}B_{21} & A_{21}B_{12} + A_{22}B_{22} \end{pmatrix}

If matrices are split into compatible blocks, the blocks multiply as if they were numbers.

See proof
  1. 1The matrices A and B are split into 2 × 2 blocks, with the columns of A cut in the same way as the rows of B.A=(A11A12A21A22),B=(B11B12B21B22)A = \begin{pmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{pmatrix},\qquad B = \begin{pmatrix} B_{11} & B_{12} \\ B_{21} & B_{22} \end{pmatrix}
  2. 2Apply row times column, but with blocks. Each product of blocks is computed as an ordinary matrix product.AB=(A11B11+A12B21A11B12+A12B22A21B11+A22B21A21B12+A22B22)AB = \begin{pmatrix} A_{11}B_{11} + A_{12}B_{21} & A_{11}B_{12} + A_{12}B_{22} \\ A_{21}B_{11} + A_{22}B_{21} & A_{21}B_{12} + A_{22}B_{22} \end{pmatrix}

What it is for. It splits an enormous product into several small ones (computers do it to share out the work) and takes advantage of blocks of zeros or identities to save computations.

07 · Trace

The trace

The trace of a square matrix is the sum of the entries on its main diagonal.

Tracetr(A) = a11 + a22 + … + ann = Σi=1n aii.
  1. 1Worked example. The diagonal of A is 1, 5, 9.A=(123456789)A = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix}
  2. 2The trace is the sum of the diagonal.tr⁡(A)=1+5+9=15\operatorname{tr}(A) = 1 + 5 + 9 = 15

Properties

The trace is linear: tr(A + B) = tr A + tr B and tr(λA) = λ tr A

tr⁡(A+B)=tr⁡A+tr⁡B,tr⁡(λA)=λtr⁡A\operatorname{tr}(A + B) = \operatorname{tr}A + \operatorname{tr}B,\qquad \operatorname{tr}(\lambda A) = \lambda \operatorname{tr}A

The trace of a sum is the sum of the traces, and scalars come out.

See proof

tr(AB) = tr(BA)

tr⁡(AB)=tr⁡(BA)\operatorname{tr}(AB) = \operatorname{tr}(BA)

The trace of a product does not depend on the order of the two factors, even though AB and BA are different.

See proof

What it is for. It summarises a square matrix in a single number and has very handy properties for simplifying. It is used in statistics, physics and machine learning.

08 · Applications

What are operations with matrices for?

Graphics and gamesRotating, scaling and moving a 3D object is multiplying its points by matrices. Chaining transformations is multiplying matrices.
Economics and logisticsIf one matrix stores quantities and another prices, their product gives the total costs at once.
Neural networksEach layer of a neural network multiplies a matrix of weights by the input data: it is all products and sums of matrices.
Systems of equationsA system is written as AX = b, and the whole solving method uses operations with matrices.
Images and soundA black and white image is a matrix of brightness values. Adding matrices blends images and multiplying by a scalar changes the brightness.
ProbabilityMarkov chains use the matrix product to predict the state of a system step by step.
Proof

Properties of the sum: commutative, associative, zero and opposite

A+B=B+A,(A+B)+C=A+(B+C)A + B = B + A,\qquad (A + B) + C = A + (B + C)
  1. [A+B]ij=aij+bij[A + B]_{ij} = a_{ij} + b_{ij}

    To add two matrices of the same size, add the entries in the same position: entry (i, j) of A + B is a_ij + b_ij.

  2. aij+bij=bij+aij  ⇒  A+B=B+Aa_{ij} + b_{ij} = b_{ij} + a_{ij} \;\Rightarrow\; A + B = B + A

    Commutative: in each position a_ij + b_ij = b_ij + a_ij, because they are numbers. Every position agrees, so A + B = B + A.

  3. [(A+B)+C]ij=(aij+bij)+cij=aij+(bij+cij)=[A+(B+C)]ij[(A + B) + C]_{ij} = (a_{ij} + b_{ij}) + c_{ij} = a_{ij} + (b_{ij} + c_{ij}) = [A + (B + C)]_{ij}

    Associative: in each position (a_ij + b_ij) + c_ij = a_ij + (b_ij + c_ij). So (A + B) + C = A + (B + C) and you can write A + B + C without parentheses.

  4. aij+0=aij  ⇒  A+0=0+A=Aa_{ij} + 0 = a_{ij} \;\Rightarrow\; A + 0 = 0 + A = A

    Zero element: the zero matrix 0 (all zeros, same size) changes nothing, because a_ij + 0 = a_ij.

  5. aij+(−aij)=0  ⇒  A+(−A)=0a_{ij} + (-a_{ij}) = 0 \;\Rightarrow\; A + (-A) = 0

    Opposite: the matrix −A has every entry with its sign changed, and a_ij + (−a_ij) = 0. So A + (−A) = 0, and subtracting A − B means adding the opposite A + (−B).

  6. (1203−14)+(205−112)=(325206)\begin{pmatrix} 1 & 2 & 0 \\ 3 & -1 & 4 \end{pmatrix} + \begin{pmatrix} 2 & 0 & 5 \\ -1 & 1 & 2 \end{pmatrix} = \begin{pmatrix} 3 & 2 & 5 \\ 2 & 0 & 6 \end{pmatrix}

    Example with 2 × 3 matrices. In the drawing, the marked position adds 4 + 2 = 6; the others are added the same way, each one with the entry in the same position.

What it's for

They guarantee that you can work with sums of matrices as with sums of numbers: reorder, group and solve for unknowns. For example, from A + X = B you get X = B − A.

Proof

Properties of the product by a scalar

λ(A+B)=λA+λB,(λ+μ)A=λA+μA,(λμ)A=λ(μA)\lambda(A + B) = \lambda A + \lambda B,\quad (\lambda + \mu)A = \lambda A + \mu A,\quad (\lambda\mu)A = \lambda(\mu A)
  1. [λA]ij=λ aij[\lambda A]_{ij} = \lambda\, a_{ij}

    Multiplying a matrix by a number (scalar) λ means multiplying every entry by λ: entry (i, j) of λA is λ·a_ij.

  2. [λ(A+B)]ij=λ(aij+bij)=λaij+λbij=[λA+λB]ij[\lambda(A + B)]_{ij} = \lambda(a_{ij} + b_{ij}) = \lambda a_{ij} + \lambda b_{ij} = [\lambda A + \lambda B]_{ij}

    λ(A + B) = λA + λB: in each position λ(a_ij + b_ij) = λa_ij + λb_ij, which is the distributive property of numbers.

  3. [(λ+μ)A]ij=(λ+μ) aij=λaij+μaij=[λA+μA]ij[(\lambda + \mu)A]_{ij} = (\lambda + \mu)\, a_{ij} = \lambda a_{ij} + \mu a_{ij} = [\lambda A + \mu A]_{ij}

    (λ + μ)A = λA + μA: in each position (λ + μ)a_ij = λa_ij + μa_ij.

  4. [(λμ)A]ij=(λμ) aij=λ(μ aij)=[λ(μA)]ij[(\lambda\mu)A]_{ij} = (\lambda\mu)\, a_{ij} = \lambda(\mu\, a_{ij}) = [\lambda(\mu A)]_{ij}

    (λμ)A = λ(μA): in each position (λμ)a_ij = λ(μ·a_ij). Multiplying by μ and then by λ is the same as multiplying by λμ.

  5. 3(1−204)=(3−6012)3 \begin{pmatrix} 1 & -2 \\ 0 & 4 \end{pmatrix} = \begin{pmatrix} 3 & -6 \\ 0 & 12 \end{pmatrix}

    Example: multiply every entry of the matrix by 3. In the drawing, each cell of 3A is three times the matching cell of A.

  6. (2+3)A=(5−10020)=(2−408)+(3−6012)=2A+3A(2 + 3)A = \begin{pmatrix} 5 & -10 \\ 0 & 20 \end{pmatrix} = \begin{pmatrix} 2 & -4 \\ 0 & 8 \end{pmatrix} + \begin{pmatrix} 3 & -6 \\ 0 & 12 \end{pmatrix} = 2A + 3A

    Check of the second property with λ = 2 and μ = 3: 5A equals 2A + 3A.

What it's for

They let you take out common factors and distribute numbers in expressions with matrices, just as in ordinary algebra. Together with the properties of the sum, they say that matrices can be treated like vectors: you can add them and multiply them by numbers.

Proof

The product of matrices: row times column

cij=∑k=1paik bkjc_{ij} = \sum_{k=1}^{p} a_{ik}\, b_{kj}
  1. (a1a2⋯ap)(b1b2⋮bp)=a1b1+a2b2+⋯+apbp\begin{pmatrix} a_1 & a_2 & \cdots & a_p \end{pmatrix} \begin{pmatrix} b_1 \\ b_2 \\ \vdots \\ b_p \end{pmatrix} = a_1 b_1 + a_2 b_2 + \dots + a_p b_p

    A row times a column (with the same number of entries, p) gives a number: multiply the entries in pairs, a₁b₁, a₂b₂, … and add them up.

  2. A∈Mn×p, B∈Mp×m  ⇒  C=AB∈Mn×mA \in M_{n \times p},\ B \in M_{p \times m} \;\Rightarrow\; C = AB \in M_{n \times m}

    To multiply, the number of columns of A must equal the number of rows of B. If A is n × p and B is p × m, the product C = AB is n × m.

  3. cij=Ai∙ B∙jc_{ij} = A_{i\bullet}\, B_{\bullet j}

    Entry c_ij of C is row i of A times column j of B.

  4. cij=ai1b1j+ai2b2j+⋯+aipbpj=∑k=1paik bkjc_{ij} = a_{i1} b_{1j} + a_{i2} b_{2j} + \dots + a_{ip} b_{pj} = \sum_{k=1}^{p} a_{ik}\, b_{kj}

    Written with the entries and the sum symbol, c_ij = a_i1·b_1j + a_i2·b_2j + … + a_ip·b_pj. The index k of the sum runs through the p terms (the notes use the same letter p for the index and for the size; here the index is k).

  5. c21=4⋅7+5⋅9+6⋅11=28+45+66=139c_{21} = 4 \cdot 7 + 5 \cdot 9 + 6 \cdot 11 = 28 + 45 + 66 = 139

    Example: c₂₁ uses row 2 of A, (4, 5, 6), and column 1 of B, (7, 9, 11). In the drawing, the row is orange, the column is teal and the result is pink: 4·7 + 5·9 + 6·11 = 139.

  6. (123456)(789101112)=(5864139154)\begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{pmatrix} \begin{pmatrix} 7 & 8 \\ 9 & 10 \\ 11 & 12 \end{pmatrix} = \begin{pmatrix} 58 & 64 \\ 139 & 154 \end{pmatrix}

    Repeating this with every row and every column gives the complete 2 × 2 matrix AB.

What it's for

The product is the way to combine information: apply one transformation after another, compute totals such as price times quantity, or write a system of equations as AX = b.

Proof

Size of the product: (n × p)(p × m) = n × m

(n×p)⋅(p×m)=(n×m)(n \times p)\cdot(p \times m) = (n \times m)
  1. A=[aik]n×p,B=[bkj]q×mA = [a_{ik}]_{n \times p},\qquad B = [b_{kj}]_{q \times m}

    Let A be n × p and B be q × m (we do not yet know whether q equals p).

  2. (ai1⋯aip)(b1j⋮bqj)  ⇒  p=q\begin{pmatrix} a_{i1} & \cdots & a_{ip} \end{pmatrix} \begin{pmatrix} b_{1j} \\ \vdots \\ b_{qj} \end{pmatrix} \;\Rightarrow\; p = q

    Each entry c_ij requires multiplying row i of A (with p entries) by column j of B (with q entries), in pairs. That is only possible if p = q: A must have as many columns as B has rows.

  3. cij (i=1,…,n; j=1,…,m)  ⇒  AB∈Mn×mc_{ij}\ (i = 1, \dots, n;\ j = 1, \dots, m) \;\Rightarrow\; AB \in M_{n \times m}

    There is one c_ij for each row i of A (n rows) and each column j of B (m columns), so AB has size n × m.

  4. (2×3)⋅(3×4)=(2×4)(2 \times 3)\cdot(3 \times 4) = (2 \times 4)

    Example: (2 × 3)·(3 × 4) = (2 × 4). The two middle 3s match (green in the drawing) and disappear; 2 and 4 remain.

  5. (3×4)⋅(2×3):4≠2(3 \times 4)\cdot(2 \times 3):\quad 4 \neq 2

    The other way round, (3 × 4)·(2 × 3) cannot be computed, because 4 ≠ 2. The existence of AB does not guarantee the existence of BA.

What it's for

It is the first check before multiplying matrices: if the sizes do not fit, the product is not defined. It also tells you the size of the result without doing any arithmetic.

Proof

Associative property: (AB)C = A(BC)

(AB)C=A(BC)(AB)C = A(BC)
  1. A∈Mn×p, B∈Mp×q, C∈Mq×m  ⇒  (AB)C, A(BC)∈Mn×mA \in M_{n \times p},\ B \in M_{p \times q},\ C \in M_{q \times m} \;\Rightarrow\; (AB)C,\ A(BC) \in M_{n \times m}

    With A being n × p, B being p × q and C being q × m, both (AB)C and A(BC) can be computed and are n × m. In the drawing, the upper bracket groups AB and the lower one groups BC.

  2. [(AB)C]il=∑k=1q(AB)ik ckl=∑k=1q(∑j=1paij bjk)ckl[(AB)C]_{il} = \sum_{k=1}^{q} (AB)_{ik}\, c_{kl} = \sum_{k=1}^{q} \Bigl(\sum_{j=1}^{p} a_{ij}\, b_{jk}\Bigr) c_{kl}

    Entry (i, l) of (AB)C is the sum over k of (AB)_ik · c_kl, and each (AB)_ik is itself a sum over j of a_ij · b_jk.

  3. [A(BC)]il=∑j=1paij (BC)jl=∑j=1paij(∑k=1qbjk ckl)[A(BC)]_{il} = \sum_{j=1}^{p} a_{ij}\, (BC)_{jl} = \sum_{j=1}^{p} a_{ij} \Bigl(\sum_{k=1}^{q} b_{jk}\, c_{kl}\Bigr)

    Entry (i, l) of A(BC) is the sum over j of a_ij · (BC)_jl, and each (BC)_jl is a sum over k of b_jk · c_kl.

  4. [(AB)C]il=[A(BC)]il=∑j=1p∑k=1qaij bjk ckl[(AB)C]_{il} = [A(BC)]_{il} = \sum_{j=1}^{p} \sum_{k=1}^{q} a_{ij}\, b_{jk}\, c_{kl}

    In both cases we get the same double sum of the terms a_ij · b_jk · c_kl, just added in a different order. Since the sum of numbers does not depend on the order, they agree.

  5. (AB)C=A(BC)=:ABC(AB)C = A(BC) =: ABC

    That is why you can write ABC without parentheses. The symbol =: means "is defined as".

  6. A=(1201), B=(0131), C=(1021):(AB)C=(12351)=A(BC)A = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix},\ B = \begin{pmatrix} 0 & 1 \\ 3 & 1 \end{pmatrix},\ C = \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix}:\quad (AB)C = \begin{pmatrix} 12 & 3 \\ 5 & 1 \end{pmatrix} = A(BC)

    Example: AB = ((6, 3), (3, 1)) and (AB)C = ((12, 3), (5, 1)). The other way, BC = ((2, 1), (5, 1)) and A(BC) = ((12, 3), (5, 1)): they match.

What it's for

It lets you choose the cheapest order of computation (for example, multiplying by a vector first) and chain transformations: applying B and then A is the same as applying a single matrix AB.

Proof

Distributive property: (A + B)C = AC + BC

(A+B)C=AC+BC,C(A+B)=CA+CB(A + B)C = AC + BC,\qquad C(A + B) = CA + CB
  1. A,B∈Mn×p, C∈Mp×mA, B \in M_{n \times p},\ C \in M_{p \times m}

    For them to be added, A and B must have the same size, n × p; C is p × m, so all three products are defined.

  2. [(A+B)C]ij=∑k=1p(aik+bik) ckj[(A + B)C]_{ij} = \sum_{k=1}^{p} (a_{ik} + b_{ik})\, c_{kj}

    Entry (i, j) of (A + B)C is the sum over k of (a_ik + b_ik) · c_kj.

  3. [(A+B)C]ij=∑k=1paik ckj+∑k=1pbik ckj=[AC]ij+[BC]ij[(A + B)C]_{ij} = \sum_{k=1}^{p} a_{ik}\, c_{kj} + \sum_{k=1}^{p} b_{ik}\, c_{kj} = [AC]_{ij} + [BC]_{ij}

    Each term is expanded with the distributive property of numbers and split into two sums: the first is entry (i, j) of AC and the second is entry (i, j) of BC. Therefore (A + B)C = AC + BC.

  4. [C(A+B)]ij=∑k=1pcik (akj+bkj)=[CA]ij+[CB]ij[C(A + B)]_{ij} = \sum_{k=1}^{p} c_{ik}\, (a_{kj} + b_{kj}) = [CA]_{ij} + [CB]_{ij}

    With C on the left the reasoning is the same: c_ik · (a_kj + b_kj) gives C(A + B) = CA + CB. C must stay on the left in both terms, because order matters.

  5. (1332)(1021)=(7372)=(5221)+(2151)\begin{pmatrix} 1 & 3 \\ 3 & 2 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} = \begin{pmatrix} 7 & 3 \\ 7 & 2 \end{pmatrix} = \begin{pmatrix} 5 & 2 \\ 2 & 1 \end{pmatrix} + \begin{pmatrix} 2 & 1 \\ 5 & 1 \end{pmatrix}

    Example: A + B = ((1, 3), (3, 2)) and (A + B)C = ((7, 3), (7, 2)). Separately, AC = ((5, 2), (2, 1)) and BC = ((2, 1), (5, 1)), which add up to the same thing.

What it's for

It lets you expand and factor expressions with matrices, as long as you take care over the order of the factors. For example, (A + B)² = A² + AB + BA + B².

Proof

The identity matrix is the neutral element: IA = AI = A

IA=AI=AIA = AI = A
  1. I=[δij],δij={1i=j0i≠jI = [\delta_{ij}],\qquad \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}

    The identity I has ones on the diagonal and zeros elsewhere. It is written with the Kronecker delta: δ_ij is 1 if i = j and 0 if i ≠ j.

  2. [InA]ij=∑k=1nδik akj=aij[I_n A]_{ij} = \sum_{k=1}^{n} \delta_{ik}\, a_{kj} = a_{ij}

    In entry (i, j) of IA, the sum of δ_ik · a_kj has only one non-zero term, the one with k = i, which equals a_ij. So IA = A.

  3. [AIm]ij=∑k=1maik δkj=aij[A I_m]_{ij} = \sum_{k=1}^{m} a_{ik}\, \delta_{kj} = a_{ij}

    Same on the other side: in the sum of a_ik · δ_kj only k = j survives and a_ij remains. So AI = A.

  4. A∈Mn×m:InA=AIm=AA \in M_{n \times m}:\quad I_n A = A I_m = A

    If A is n × m, the identity of order n is used on the left and the one of order m on the right. They are different identities, but the result is the same A.

  5. (1001)(123456)=(123456)=(123456)(100010001)\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{pmatrix} = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{pmatrix} = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}

    Example with a 2 × 3 matrix A: on the left, the 2 × 2 identity; on the right, the 3 × 3 identity. In the drawing, the identity has its diagonal highlighted.

What it's for

The identity plays the role of the number 1: it changes nothing when multiplying. It is essential to define the inverse matrix, which is the one that gives I when multiplied by A.

Proof

The product is not commutative: AB ≠ BA

AB≠BAAB \neq BA
  1. A∈Mn×p, B∈Mp×n  ⇒  AB∈Mn×n, BA∈Mp×pA \in M_{n \times p},\ B \in M_{p \times n} \;\Rightarrow\; AB \in M_{n \times n},\ BA \in M_{p \times p}

    Even if both AB and BA can be computed, they may have different sizes: with A being n × p and B being p × n, AB is n × n and BA is p × p. If n ≠ p they cannot be equal.

  2. A∈M2×3, B∈M3×2:AB=(5257),BA=(120552936)A \in M_{2 \times 3},\ B \in M_{3 \times 2}:\quad AB = \begin{pmatrix} 5 & 2 \\ 5 & 7 \end{pmatrix},\quad BA = \begin{pmatrix} 1 & 2 & 0 \\ 5 & 5 & 2 \\ 9 & 3 & 6 \end{pmatrix}

    Example with A being 2 × 3 and B being 3 × 2: AB is 2 × 2 and BA is 3 × 3.

  3. A=(1234), B=(0110):AB=(2143),BA=(3412)A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix},\ B = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}:\quad AB = \begin{pmatrix} 2 & 1 \\ 4 & 3 \end{pmatrix},\quad BA = \begin{pmatrix} 3 & 4 \\ 1 & 2 \end{pmatrix}

    Even with square matrices of the same size the result differs. Here B swaps: AB swaps the columns of A and BA swaps its rows.

  4. (AB)11=1⋅0+2⋅1=2≠3=0⋅1+1⋅3=(BA)11(AB)_{11} = 1 \cdot 0 + 2 \cdot 1 = 2 \neq 3 = 0 \cdot 1 + 1 \cdot 3 = (BA)_{11}

    A single entry is enough: (AB)₁₁ = 1·0 + 2·1 = 2, but (BA)₁₁ = 0·1 + 1·3 = 3. In the drawing, those cells are marked.

  5. IA=AI,A2A=AA2,(λI)A=A(λI)IA = AI,\qquad A^2 A = A A^2,\qquad (\lambda I) A = A (\lambda I)

    Some pairs do commute: the identity with any matrix, a matrix with its own powers, or a matrix with a multiple of the identity.

What it's for

It is the most frequent mistake when working with matrices. For example, (A + B)² is not A² + 2AB + B², because AB and BA are different. When solving matrix equations you must always multiply on the same side in both members.

Proof

Block multiplication

(A11A12A21A22)(B11B12B21B22)=(A11B11+A12B21A11B12+A12B22A21B11+A22B21A21B12+A22B22)\begin{pmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{pmatrix} \begin{pmatrix} B_{11} & B_{12} \\ B_{21} & B_{22} \end{pmatrix} = \begin{pmatrix} A_{11}B_{11} + A_{12}B_{21} & A_{11}B_{12} + A_{12}B_{22} \\ A_{21}B_{11} + A_{22}B_{21} & A_{21}B_{12} + A_{22}B_{22} \end{pmatrix}
  1. Aik:ni×pk,Bkj:pk×mjA_{ik}: n_i \times p_k,\qquad B_{kj}: p_k \times m_j

    Split A into blocks A_ik and B into blocks B_kj so that the partition of the columns of A matches the partition of the rows of B. Then every product A_ik·B_kj is defined. In the drawing, the dashed lines mark the cuts.

  2. cij=∑k=1paik bkj=∑k=1p1aik bkj+∑k=p1+1paik bkjc_{ij} = \sum_{k=1}^{p} a_{ik}\, b_{kj} = \sum_{k=1}^{p_1} a_{ik}\, b_{kj} + \sum_{k=p_1+1}^{p} a_{ik}\, b_{kj}

    Why it works: in the sum c_ij = Σ a_ik·b_kj, separate the first p₁ terms (those using the columns of the first block of A and the rows of the first block of B) from the remaining p − p₁ (those of the second block).

  3. Cij=Ai1B1j+Ai2B2jC_{ij} = A_{i1} B_{1j} + A_{i2} B_{2j}

    Each block of the result is the sum of the products of the blocks in row i of A by the blocks in column j of B: C_ij = A_i1·B_1j + A_i2·B_2j. It is the "row times column" rule with blocks instead of numbers; the statement shows the complete result.

  4. A=(1210340100200002), B=(1101011010122101)A = \begin{pmatrix} 1 & 2 & 1 & 0 \\ 3 & 4 & 0 & 1 \\ 0 & 0 & 2 & 0 \\ 0 & 0 & 0 & 2 \end{pmatrix},\ B = \begin{pmatrix} 1 & 1 & 0 & 1 \\ 0 & 1 & 1 & 0 \\ 1 & 0 & 1 & 2 \\ 2 & 1 & 0 & 1 \end{pmatrix}

    Example with 4 × 4 matrices split into 2 × 2 blocks. In A, the lower left block is 0 and the lower right block is 2I; the upper right block is I.

  5. C11=(1234)(1101)+(1001)(1021)=(1337)+(1021)=(2358)C_{11} = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} + \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 3 \\ 3 & 7 \end{pmatrix} + \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} = \begin{pmatrix} 2 & 3 \\ 5 & 8 \end{pmatrix}

    C₁₁ = A₁₁B₁₁ + A₁₂B₂₁. Compute the two 2 × 2 products and add them.

  6. C12=(1234)(0110)+(1001)(1201)=(2143)+(1201)=(3344)C_{12} = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} + \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 2 & 1 \\ 4 & 3 \end{pmatrix} + \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 3 & 3 \\ 4 & 4 \end{pmatrix}

    C₁₂ = A₁₁B₁₂ + A₁₂B₂₂, computed in the same way.

  7. C21=0⋅B11+2B21=(2042),C22=0⋅B12+2B22=(2402)C_{21} = 0 \cdot B_{11} + 2 B_{21} = \begin{pmatrix} 2 & 0 \\ 4 & 2 \end{pmatrix},\qquad C_{22} = 0 \cdot B_{12} + 2 B_{22} = \begin{pmatrix} 2 & 4 \\ 0 & 2 \end{pmatrix}

    In the bottom row A₂₁ = 0, so only the second term remains: C₂₁ = 2B₂₁ and C₂₂ = 2B₂₂.

  8. AB=(2333584420244202)AB = \begin{pmatrix} 2 & 3 & 3 & 3 \\ 5 & 8 & 4 & 4 \\ 2 & 0 & 2 & 4 \\ 4 & 2 & 0 & 2 \end{pmatrix}

    Putting the four blocks together gives AB. It is exactly the matrix you get by multiplying the two 4 × 4 matrices in the ordinary way.

What it's for

It reduces big matrices to operations with small ones, and blocks of zeros save a great deal of computation. It is used in computers (splitting the work into pieces) and in proofs about matrices with structure.

Proof

The trace is linear: tr(A + B) = tr A + tr B and tr(λA) = λ tr A

tr⁡(A+B)=tr⁡A+tr⁡B,tr⁡(λA)=λtr⁡A\operatorname{tr}(A + B) = \operatorname{tr}A + \operatorname{tr}B,\qquad \operatorname{tr}(\lambda A) = \lambda \operatorname{tr}A
  1. tr⁡A=a11+a22+⋯+ann=∑i=1naii\operatorname{tr}A = a_{11} + a_{22} + \dots + a_{nn} = \sum_{i=1}^{n} a_{ii}

    The trace of a square matrix is the sum of the entries on its main diagonal: tr A = a₁₁ + a₂₂ + … + a_nn.

  2. tr⁡(A+B)=∑i=1n(aii+bii)=∑i=1naii+∑i=1nbii\operatorname{tr}(A + B) = \sum_{i=1}^{n} (a_{ii} + b_{ii}) = \sum_{i=1}^{n} a_{ii} + \sum_{i=1}^{n} b_{ii}

    The diagonal of A + B has the entries a_ii + b_ii. Adding them, they split into the sum of the a_ii and the sum of the b_ii.

  3. tr⁡(A+B)=tr⁡A+tr⁡B\operatorname{tr}(A + B) = \operatorname{tr}A + \operatorname{tr}B

    That is, tr(A + B) = tr A + tr B.

  4. tr⁡(λA)=∑i=1nλ aii=λ∑i=1naii=λtr⁡A\operatorname{tr}(\lambda A) = \sum_{i=1}^{n} \lambda\, a_{ii} = \lambda \sum_{i=1}^{n} a_{ii} = \lambda \operatorname{tr}A

    With a scalar, each diagonal entry is multiplied by λ, and λ comes out as a common factor of the sum: tr(λA) = λ·tr A.

  5. tr⁡(6246)=6+6=12=5+7=tr⁡(1234)+tr⁡(5012)\operatorname{tr}\begin{pmatrix} 6 & 2 \\ 4 & 6 \end{pmatrix} = 6 + 6 = 12 = 5 + 7 = \operatorname{tr}\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} + \operatorname{tr}\begin{pmatrix} 5 & 0 \\ 1 & 2 \end{pmatrix}

    Example: A and B have traces 1 + 4 = 5 and 5 + 2 = 7. A + B has diagonal 6 and 6, with trace 12 = 5 + 7. In the drawing, the diagonal is highlighted.

What it's for

It lets you compute traces of complicated expressions by adding the traces of each piece. The trace summarises a square matrix in a single number.

Proof

tr(AB) = tr(BA)

tr⁡(AB)=tr⁡(BA)\operatorname{tr}(AB) = \operatorname{tr}(BA)
  1. A∈Mn×p, B∈Mp×n  ⇒  AB∈Mn×n, BA∈Mp×pA \in M_{n \times p},\ B \in M_{p \times n} \;\Rightarrow\; AB \in M_{n \times n},\ BA \in M_{p \times p}

    With A being n × p and B being p × n, AB is n × n and BA is p × p. Both are square, so both have a trace, although they are not the same size.

  2. tr⁡(AB)=∑i=1n(AB)ii=∑i=1n∑k=1paik bki\operatorname{tr}(AB) = \sum_{i=1}^{n} (AB)_{ii} = \sum_{i=1}^{n} \sum_{k=1}^{p} a_{ik}\, b_{ki}

    The trace of AB adds the diagonal entries (AB)_ii, and each of them is itself the sum of the products a_ik·b_ki. The result is a double sum of all the a_ik·b_ki.

  3. tr⁡(BA)=∑k=1p(BA)kk=∑k=1p∑i=1nbki aik\operatorname{tr}(BA) = \sum_{k=1}^{p} (BA)_{kk} = \sum_{k=1}^{p} \sum_{i=1}^{n} b_{ki}\, a_{ik}

    The trace of BA adds the (BA)_kk, and each of them is the sum of b_ki·a_ik. It is another double sum, with the same products.

  4. ∑i=1n∑k=1paik bki=∑k=1p∑i=1nbki aik  ⇒  tr⁡(AB)=tr⁡(BA)\sum_{i=1}^{n} \sum_{k=1}^{p} a_{ik}\, b_{ki} = \sum_{k=1}^{p} \sum_{i=1}^{n} b_{ki}\, a_{ik} \;\Rightarrow\; \operatorname{tr}(AB) = \operatorname{tr}(BA)

    The terms are the same (b_ki·a_ik = a_ik·b_ki) and only the order of adding them changes, which does not matter. Therefore tr(AB) = tr(BA).

  5. A=(120312), B=(102103):tr⁡(AB)=5+7=12=1+5+6=tr⁡(BA)A = \begin{pmatrix} 1 & 2 & 0 \\ 3 & 1 & 2 \end{pmatrix},\ B = \begin{pmatrix} 1 & 0 \\ 2 & 1 \\ 0 & 3 \end{pmatrix}:\quad \operatorname{tr}(AB) = 5 + 7 = 12 = 1 + 5 + 6 = \operatorname{tr}(BA)

    Example with A being 2 × 3 and B being 3 × 2: AB (2 × 2) has trace 5 + 7 = 12 and BA (3 × 3) has trace 1 + 5 + 6 = 12, even though they are matrices of different sizes and do not look alike.

  6. tr⁡(ABC)=tr⁡(BCA)=tr⁡(CAB)\operatorname{tr}(ABC) = \operatorname{tr}(BCA) = \operatorname{tr}(CAB)

    With three factors they can only be moved in a cycle: tr(ABC) = tr(BCA) = tr(CAB). In general, tr(ACB) may be different.

What it's for

It is a widely used tool for simplifying expressions with matrices and proving properties. It appears in statistics, quantum mechanics and machine learning.

Found this useful?

This guide is free and ad-free. If it helps you study, you can buy me a coffee.

Buy me a coffee