🔗 Singular Value Decomposition

🔗 Mathematics

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix that generalizes the eigendecomposition of a square normal matrix to any ${\displaystyle m\times n}$ matrix via an extension of the polar decomposition.

Specifically, the singular value decomposition of an ${\displaystyle m\times n}$ real or complex matrix ${\displaystyle \mathbf {M} }$ is a factorization of the form ${\displaystyle \mathbf {U\Sigma V^{*}} }$, where ${\displaystyle \mathbf {U} }$ is an ${\displaystyle m\times m}$ real or complex unitary matrix, ${\displaystyle \mathbf {\Sigma } }$ is an ${\displaystyle m\times n}$ rectangular diagonal matrix with non-negative real numbers on the diagonal, and ${\displaystyle \mathbf {V} }$ is an ${\displaystyle n\times n}$ real or complex unitary matrix. If ${\displaystyle \mathbf {M} }$ is real, ${\displaystyle \mathbf {U} }$ and ${\displaystyle \mathbf {V} =\mathbf {V^{*}} }$ are real orthonormal matrices.

The diagonal entries ${\displaystyle \sigma _{i}=\Sigma _{ii}}$ of ${\displaystyle \mathbf {\Sigma } }$ are known as the singular values of ${\displaystyle \mathbf {M} }$. The number of non-zero singular values is equal to the rank of ${\displaystyle \mathbf {M} }$. The columns of ${\displaystyle \mathbf {U} }$ and the columns of ${\displaystyle \mathbf {V} }$ are called the left-singular vectors and right-singular vectors of ${\displaystyle \mathbf {M} }$, respectively.

The SVD is not unique. It is always possible to choose the decomposition so that the singular values ${\displaystyle \Sigma _{ii}}$are in descending order. In this case, Σ (but not always U and V) is uniquely determined by M.

The term sometimes refers to the compact SVD, a similar decomposition ${\displaystyle \mathbf {M} =\mathbf {U\Sigma V^{*}} }$ in which Σ is square diagonal of size ${\displaystyle r\times r}$, where ${\displaystyle r\leq \min\{m,n\}}$ is the rank of M, and has only the non-zero singular values. In this variant, ${\displaystyle \mathbf {U} }$ is an ${\displaystyle m\times r}$ matrix and ${\displaystyle \mathbf {V} }$ is an ${\displaystyle n\times r}$ matrix, such that ${\displaystyle \mathbf {U^{*}U} =\mathbf {V^{*}V} =\mathbf {I} _{r\times r}}$.

Mathematical applications of the SVD include computing the pseudoinverse, matrix approximation, and determining the rank, range, and null space of a matrix. The SVD is also extremely useful in all areas of science, engineering, and statistics, such as signal processing, least squares fitting of data, and process control.