Gilbert Strang, Linear Algebra and Its Applications
more references:
APPLIED LINEAR ALGEBRA
B. Noble, "Applied Linear Algebra" 1969
NUMERICAL LINEAR ALGEBRA
G. Forsythe and C. Moler, "Computer Solution of Linear Algebraic Systems" 1967
C. L. Lawson and R. J. Hanson, "Solving Least Squares Problems" 1974
G. W. Stewart, "Introduction to Matrix Computations", 1973
R. S. Varga, "Matrix Iterative Analysis", 1962
J. M. Wilkinson, "Rounding Errors in Algebraic Processes", 1963
J. M. Wilkinson, "The Algebraic Eigenvalue Problem", 1965
J. M. Wikinson and C. Reinsch, eds, "Handbook for Automatic Computation II, Linear Algebra", Springer, 1971
D. M. Young, "Iterative Solution of Large Linear Systems", 1971
Gilbert Strang
Linear Algebra and Its Applications
1976, 1st edition
Strang was at MIT
Before forget, let me say that Strang does talk some about Regression Analysis, Factor Analysis, and Principle Component Analysis
So he starts out explaining that primarily linear algebra is about simultaneous equations and Gaussian elimination. The second idea will be determinants and Cramer's rule.
He will show Gaussian Elimination and talk about zero pivots and when you have a singular matrix. It will get into LU factorization, which results from Gaussian Elimination. And then you use back substitution.
Tends to be n^2 operations for Gaussian Elimination.
So he talks about Matrix Multiplication.
So you will be doing Gaussian Elimination, and you will be logging the results in an Elementary Matrix, E.
So you will premultiply Ax with E, on both sides, and matrix multiplication is associative.
So you get this upper triangular matrix, and these various E matrices which log what was done to get it, cause you will want to back substitute for your solution.
Usually you will want to do row substitution in order to get bigger pivots, to avoid zero, but also to minimize round off errors. I think it is actually biggest ABS pivot.
And so a P matrix to log this is introduced, Permutation Matrix. You also premultiply both sides by this.
So you are finding the inverse of the original A matrix. ( the other way of doing this, Determinant and Adujunct Matrices, is extremely slow )
So you use the Gauss-Jordan method:
https://en.wikipedia.org/wiki/Gaussian_eliminationAnd then he talks about Band Matrices, which are matrices where the only non-zero elements are close to the diagonal. I guess this is a particular form of the Sparse Matrix
Good links on this:
Sparse Matrix
https://en.wikipedia.org/wiki/Sparse_matrixBand Matrix
https://en.wikipedia.org/wiki/Band_matrixHe looks at a differential equation with a two point boundary condition. A geometrical spacer, h, is introduced. This makes the problem discrete, and how small h is determines the number of equations and the number of unknowns, and of course this results in a band matrix.
So he goes into a more formal theory of simultaneous linear equations, and gets into Vector Spaces and Subspaces, and he goes into graphic representations.
And so we are talking about the row space of A, the nullspace of A, the column space of A, and the left nullspace of A.
Talks about Orthogonality of Vectors and Subspaces.
Fundamental Theorem of Linear Algebra, Part 1 and Part 2.
So, talks about Orthogonal Projections and Least Squares, and starts off talking about Inner Products and Transposes, and the Schwarz Inequality
Projections onto Subspaces and Least Squares
Least Squares solution satisfies "normal equations".
Projection Matrices, P
Least Squares Fitting of Data
Orthogonal Bases, Orthogonal Martices, And Gram-Schmidt Orthogonalization
Hilbert Space
Fourier Series
Legendre Polynomials
Pseudoinverse and the Singular Value Decomposition
Weighted Least Squares
now there is a big change in the book as he shifts to the discussion of Determinants
pg 146:
"The determinant provides and explicit "formula," a concise and definite expression in closed form, for quantities such as A^-1"
gives test for invertibility
gives volume of parallelepiped
Jacobian Determinant
Gilbert Strang, Linear Algebra and Its Applications
more references:
APPLIED LINEAR ALGEBRA
B. Noble, "Applied Linear Algebra" 1969
NUMERICAL LINEAR ALGEBRA
G. Forsythe and C. Moler, "Computer Solution of Linear Algebraic Systems" 1967
C. L. Lawson and R. J. Hanson, "Solving Least Squares Problems" 1974
G. W. Stewart, "Introduction to Matrix Computations", 1973
R. S. Varga, "Matrix Iterative Analysis", 1962
J. M. Wilkinson, "Rounding Errors in Algebraic Processes", 1963
J. M. Wilkinson, "The Algebraic Eigenvalue Problem", 1965
J. M. Wikinson and C. Reinsch, eds, "Handbook for Automatic Computation II, Linear Algebra", Springer, 1971
D. M. Young, "Iterative Solution of Large Linear Systems", 1971
Gilbert Strang
Linear Algebra and Its Applications
1976, 1st edition
Strang was at MIT
Before forget, let me say that Strang does talk some about Regression Analysis, Factor Analysis, and Principle Component Analysis
So he starts out explaining that primarily linear algebra is about simultaneous equations and Gaussian elimination. The second idea will be determinants and Cramer's rule.
He will show Gaussian Elimination and talk about zero pivots and when you have a singular matrix. It will get into LU factorization, which results from Gaussian Elimination. And then you use back substitution.
Tends to be n^2 operations for Gaussian Elimination.
So he talks about Matrix Multiplication.
So you will be doing Gaussian Elimination, and you will be logging the results in an Elementary Matrix, E.
So you will premultiply Ax with E, on both sides, and matrix multiplication is associative.
So you get this upper triangular matrix, and these various E matrices which log what was done to get it, cause you will want to back substitute for your solution.
Usually you will want to do row substitution in order to get bigger pivots, to avoid zero, but also to minimize round off errors. I think it is actually biggest ABS pivot.
And so a P matrix to log this is introduced, Permutation Matrix. You also premultiply both sides by this.
So you are finding the inverse of the original A matrix. ( the other way of doing this, Determinant and Adujunct Matrices, is extremely slow )
So you use the Gauss-Jordan method:
https://en.wikipedia.org/wiki/Gaussian_eliminationAnd then he talks about Band Matrices, which are matrices where the only non-zero elements are close to the diagonal. I guess this is a particular form of the Sparse Matrix
Good links on this:
Sparse Matrix
https://en.wikipedia.org/wiki/Sparse_matrixBand Matrix
https://en.wikipedia.org/wiki/Band_matrixHe looks at a differential equation with a two point boundary condition. A geometrical spacer, h, is introduced. This makes the problem discrete, and how small h is determines the number of equations and the number of unknowns, and of course this results in a band matrix.
So he goes into a more formal theory of simultaneous linear equations, and gets into Vector Spaces and Subspaces, and he goes into graphic representations.
And so we are talking about the row space of A, the nullspace of A, the column space of A, and the left nullspace of A.
Talks about Orthogonality of Vectors and Subspaces.
Fundamental Theorem of Linear Algebra, Part 1 and Part 2.
So, talks about Orthogonal Projections and Least Squares, and starts off talking about Inner Products and Transposes, and the Schwarz Inequality
Projections onto Subspaces and Least Squares
Least Squares solution satisfies "normal equations".
Projection Matrices, P
Least Squares Fitting of Data
Orthogonal Bases, Orthogonal Martices, And Gram-Schmidt Orthogonalization
Hilbert Space
Fourier Series
Legendre Polynomials
Pseudoinverse and the Singular Value Decomposition
Weighted Least Squares
now there is a big change in the book as he shifts to the discussion of Determinants
pg 146:
"The determinant provides and explicit "formula," a concise and definite expression in closed form, for quantities such as A^-1"
gives test for invertibility
gives volume of parallelepiped
Jacobian Determinant
https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinantfinding determinants needs n! computations!!!
Cramer's Rule
Expansion in cofactors, and finding inverse from adjugate matrix
Then book makes big shift to Eigenvalues and Eigenvectors
https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinantfinding determinants needs n! computations!!!
Cramer's Rule
Expansion in cofactors, and finding inverse from adjugate matrix
Then book makes big shift to Eigenvalues and Eigenvectors