Similarity and Diagonalization

In the last section we saw that diagonal and triangular matrices are nice, in the sense that we can easily find their eigenvalues (they are the entries on the diagonal).  It would be nice if we could relate a given square matrix to a diagonal or triangular one in a way such that they have the same eigenvalues.  Of course, we know that row reduction, gives us a triangular matrix.  However, row reduction does not preserve eigenvalues.  In this section, we consider a different sort of transformation of a matrix that does behave well with respect to eigenvalues.

Similar Matrices

Definition

Let A and B be n \times n matrices.  We say that A is similar to B if there is an n \times n matrix P such that P^{-1}AP = B.  

If A is similar to B we write A \sim B.

Example:  Let A = \left[ \begin{array}{cc} 1 & 1 \\ 2 & -1 \end{array} \right] and B = \left[ \begin{array}{cc} 2& -1 \\ 1 & -2 \end{array} \right].

Then A \sim B since if P = \left[ \begin{array}{cc} 1 & -1\\0 & 2 \end{array} \right], then P^{-1}AP = B.

Remarks:

In the previous example, we didn't need to compute to see that A \sim B.  Instead we could have check that AP = PB.

The matrix P depends on A and B.  It is not unique for a given pair of similar matrices A and B (i.e., there could be more than one matrix, you could have P_1 and P_2 both work.)  To see this, simply take A = B = I, in which case I \sim I, since P^{-1} I P = I for any invertible matrix P.

Theorem

Let A,B, and C be n \times n matrices.

a. A \sim A

b. If A \sim B, then B \sim A

c. If A \sim B and B \sim C, then A \sim C.

Remark:  Any relation satisfying the three properties of this theorem is called an equivalence relation.  Equivalence relations arise frequently in mathematics, and objects that are related via an equivalence relation usually share important properties.  We are about to see that this is true of similar matrices.

Theorem

Let A and B be n \times n matrices with A \sim B.  Then 

a. det A = det B

b. A is invertible if and only if B is invertible

c. A and B have the same rank

d. A and B have the same characteristic polynomial

e. A and B have the same eigenvalues.

Remark: Two matricies may have all these properties in common and yet still not be similar.  For example, 

A = \left[\begin{array}{cc} 1 & 0\\ 0 & 1\end{array} \right] and B = \left[ \begin{array}{cc} 1 & 1 \\ 0 & 1 \end{array} \right] both have determinant 1 and rank 2, are invertible, and have characteristic polynomial (1-\lambda)^2 and eigenvalues \lambda_1=\lambda_2 = 1.  But A is not similar to B since P^{-1}AP = I \neq B for any invertible matrix P.

This theorem is most useful in showing that two matrices are not similarly, since A and B cannot be similar if any of the properties (a) through (e) fail.

Example:

(a) A = \left[ \begin{array}{cc} 1 & 2 \\ 2 & 1\end{array} \right] and B = \left[ \begin{array}{cc} 2 & 1 \\ 1 & 2 \end{array} \right] are not similar, since det A = -3 and det B = 3.

(b) A = \left[ \begin{array}{cc} 1 & 3 \\ 2 & 2 \end{array}\right] and B = \left[ \begin{array}{cc} 1 & 1 \\ 3& -1 \end{array} \right] are not similar, since the characteristic polynomial of A is \lambda^2 - 3 \lambda - 4 and the characteristic polynomial of B is \lambda^2 -4.  Note that A and B have the same determinant and rank.

Diagonalization

The best possible situation is when a square matrix is similar to a diagonal matrix.  As you are about the to see, whether a matrix is diagonalizable is closely related to the eigenvalues and eigenvectors of the matrix.

Definition

An n \times n matrix A is diagonalizable if there is a diagonal matrix D such that A is similar to D--that is, if there is an invertible n \times n matrix P such that P^{-1}AP = D.

This is all very good, but how on earth do we find D and P?  Or even, when can this happen?

Theorem

Let A be an n \times n matrix.  Then A is diagonalizable if and only if A has n linearly independent eigenvectors.

More precisely, there exists an invertible matrix P and a diagonal matrix D such that P^{-1}AP=D if and only if the columns of P are n linearly independent eigenvectors of A and the diagonal entries of D are the eigenvalues of A corresponding to the eienvectors in P in the same order.

Example: If possible, find a matrix P that diagonalizes A = \left[\begin{array}{rrr} -2 & -4 & 1 \\ 1 & 2 & 0 \\ -5 & -8 & 4 \end{array}\right].

Here the eigenvectors are \lambda_1 = 2, \lambda_2 = 1 and \lambda_3 = 1.  

The eigenspaces have the following bases:

For \lambda_2 = \lambda_3 = 1, E_1 had basis \left[ \begin{array}{c} 1\\-1\\-1\end{array} \right].

For \lambda_1 = 2, E_2 has a basis \left[ \begin{array}{c} 0 \\ 1 \\ 4 \end{array}\right].

Since all other eigenvectors are just multiples of one of these two basis vectors, there cannot be three linearly independent eigenvectors.  Therefore, A is not diagonalizable.

Exercise: If possible, find P that diagonalizes A = \left[\begin{array}{rrr} 0 & 5 & -5 \\ 0 & -1 & 1 \\ 0 & 1 & -1 \end{array}\right].

Here the eigenvectors are \lambda_1 = \lambda_2 = 0 and \lambda_3 = -2.

For \lambda_1 = \lambda_2 = 0, E_0 has basis \left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right], \left[ \begin{array}{c} 0 \\ 1 \\ 1\end{array} \right]

For \lambda_3 = -2, E_{-2} has basis \left[ \begin{array}{c} 1 \\ -1/5 \\ 1/5 \end{array} \right].

It is straightforward to check that these three vectors are linearly independent (for example, form the matrix P = \left[ \begin{array}{ccc} 1 & 0 & 1 \\ 0 & 1 & -1/5 \\ 0 & 1 & 1/5\end{array} \right] and row reduce).  Thus the matrix P is invertible and P^{-1} A P = \left[ \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & -2 \end{array} \right] = D.

Reminder:  If you are checking by hand it is much easier to check AP = PD.

Remarks:  

When there are enough eigenvectors, they can be placed into the columns of P in any order.  However, this affects the order which the eigenvalues appear on the diagonal of D.  So make sure to put the eigenvalues on the diagonal of D in the same order you place the corresponding eigenvectors in P.

Theorem

Let A be an n \times n matrix and let \lambda_1,\lambda_2,...,\lambda_k be distinct eigenvalues of A.  

If B_i is a basis for the eigenspace E_{\lambda_i}, then B = B_1 \cup B_2 \cup \cdots \cup B_k (i.e., the total collection of basis vectors for all of the eigenspaces) is linearly independent.

Theorem

If A is an n \times n matrix with n distinct eigenvalues, then A is diagonalizable.

(Note: this is not an if and only if, as we have already seen a 3 \times 3 diagonalizable matrix with only two distinct eigenvalues)

Exaple: The matrix A = \left[ \begin{array}{cccc} 2 & 1 & 3 & -1 \\ 0 & -2 & 0 & 0 \\ 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & -1 \end{array} \right] is diagonalizable as it has 4 distinct eigenvalues.

Lemma

If A is an n \times n matrix, then the geometric multiplicity of each eigenvalue is less than or equal to its algebraic multiplicity.

Theorem: The Diagonalization Theorem

Let A be an n \times n matrix whose distinct eigenvalues are \lambda_1,\lambda_2,...,\lambda_k.  The following statements are equivalent.

a.  A is diagonalizable

b. The union B of the bases of the eigenspaces of A contains n vectors

c. The algebraic multiplicity of each eigenvalue equals its geometric multiplicity.

Looking back at two examples:

A = \left[\begin{array}{rrr} -2 & -4 & 1 \\ 1 & 2 & 0 \\ -5 & -8 & 4 \end{array}\right]

had eigenvalues \lambda_1 = 2, \lambda_2 = \lambda_3 =1 with algebraic multiplicities 1 and 2 respectively and geometric multiplicities 1 and 1.

A = \left[\begin{array}{rrr} 0 & 5 & -5 \\ 0 & -1 & 1 \\ 0 & 1 & -1 \end{array}\right]

had eigenvalues \lambda_1 = \lambda_2 = 0 and \lambda_3 = -2 with algebraic multiplicities 2 and 1 respectively and geometric multiplicities 2 and 1.

Challenge:

Compute A = \left[\begin{array}{rrr} 0 & 5 & -5 \\ 0 & -1 & 1 \\ 0 & 1 & -1 \end{array}\right]^{101}.