Analysis and Linear Algebra: The Singular Value Decomposition and Applications

This book provides an elementary analytically
inclined journey to a fundamental result of linear algebra: the
Singular Value Decomposition (SVD). SVD is a workhorse in many
applications of linear algebra to data science. Four important
applications relevant to data science are considered throughout the
book: determining the subspace that “best” approximates a
given set (dimension reduction of a data set); finding the
“best” lower rank approximation of a given matrix
(compression and general approximation problems); the Moore-Penrose
pseudo-inverse (relevant to solving least squares problems); and the
orthogonal Procrustes problem (finding the orthogonal transformation
that most closely transforms a given collection to a given
configuration), as well as its orientation-preserving version.
The point of view throughout is analytic. Readers are assumed to
have had a rigorous introduction to sequences and continuity. These
are generalized and applied to linear algebraic ideas. Along the way
to the SVD, several important results relevant to a wide variety of
fields (including random matrices and spectral graph theory) are
explored: the Spectral Theorem; minimax characterizations of
eigenvalues; and eigenvalue inequalities. By combining analytic and
linear algebraic ideas, readers see seemingly disparate areas
interacting in beautiful and applicable ways.