29. QR decomposition by Householder transformation
30. Givens Rotations
31. 雅可比(Jacobi)迭代法
32. 高斯一赛德尔(Gauss-Seidel)迭代法
33. 逐次超松弛(Succesive Over Relaxation, SOR)迭代法
34. Jacobi法和GS法的收敛性
35. 二次型(quadratic form)
36. 误差(Error)和余量(Residual)
37. 范数
38. 梯度下降法
39. 特征值和特征向量迭代的收敛意义
40. Convergence of steepest descent
41. 收敛误差和余项
42. 收敛率
43. 定义共轭(conjugate)或者又称为A-正交(A-orthogonal)
44. 共轭梯度法(P150)
45. 共轭梯度法(算法流程图)
46. 预处理共轭梯度法
47. Quiz
47. $X^TAX$求导
48. $x^Ty$求导
49. 线性方程组模型
50. $L_2$范数求解 & KKT条件(Minimum $L_2$ Norm)
51. 拉格朗日乘子算法
52. Minimum $\mathrm{L}_{2}$ Norm Solution
53. Minimum $\mathrm{L}_{p}$ Norm Solution
54. 凸性(Convexity)
54. Quiz
55. $\mathbf{L}_{0}$ norm vs. $L_{1}$ norm
55. Quiz
56. 线性规划问题
57. 二次规划问题
58. Smooth Reformulation Tricks
59. 字典学习(Dictionary Learning)
60. K-SVD
61. LASSO回归和Ridge回归
62. F-范数(Frobenius norm)
63. Trace(迹) + 矩阵求导
64. 通过矩阵Trace机制求导$\min _{A}\|A X-B\|_{F}^{2}$
65. 矩阵直接求导$\min _{A}\|A X-B\|_{F}^{2}$
66. 链式求导
67. PCA
68. SVD
68. Quiz
69. SVD分解求PCA法
70. SVD和最小二乘法的关联
71. 零空间和左零空间
72. 特征值和优化问题的关联
73. 拉普拉斯矩阵(Laplacian matrix)
74. 可逆(invertible)矩阵不会影响其他矩阵的列空间维度
75. 正交三角分解(QR分解)
76. Gram-Schmidt orthogonalization
77. 张量的秩
78. 张量分解
79. PARAFAC Decomposition
80. 常微分(Ordinary differential equations, ODE)-单变量
81. 偏微分(Partial differential equations, PDE)-多变量
82. 泰勒展开(Taylor Expansions)
83. Euler Method
84. Runge-Kutta Methods
84. Quiz
85. Adams Method
86. 高阶常微分方程(Higher Order Differential Equations)
87. 误差分析
88. Discretization Error
89. Round-off Error (考虑计算机误差)
90. Step-size
91. Stability
92. Boundary value problem (边值问题)
93. The Shooting Method
94. The Direct Method
95. Quiz
96. Nonlinear Problems
97. 有限差分法(Finite differences)
98. 散度(div)
99. 欧拉方程(Euler Equation)
100. 边界条件(Boundary Conditions)
101. 泊松方程(Poisson Equation)-PPT
102. 泊松方程 - 百度百科
103. 泊松方程的导出
104. 泊松方程在图像处理中的应用
105. 泊松方程的具体计算
106. Guass公式
107. 热传导方程(Heat diffusion equations)
108. Parabolic equation(抛物线方程)
109. 波动方程(Wave equations)
110. 波动方程的导出
111. 双曲线方程(Hyperbolic equation)
112. 对流扩散方程(Advection-Diffusion Equation)
113. Typical PDEs
114. Euler-Lagrange equations
115. Basics of Fourier Series
116. Spectral methods (频谱法)
117. bais vectors
118. $e^{ix}$
119. 和差化积公式
120. 可视化傅里叶域性质
121. 傅里叶矩阵
123. 快速傅里叶变换(Fast Fourier Transform)
124. 傅里叶变换与导数的关系
125. Spectral Methods
126. Poisson Equations
127. Heat Diffusion Equations
128. 伪谱法(Filtered Pseudo-Spectral)
129. 有限差分法和谱方法的对比
# 知识点-2019期测试
**Q1.** Suppose $A \in R^{m \times n}$ has full rank, where $m<n$. Show that taking $x=A^{T}\left(A A^{T}\right)^{-1} b$ solves the following optimization problem:
**Q2.** Suppose $A \in R^{m \times n}$ and $B \in R^{m \times k}$ are given, where $m \geq n$ and $A$ is full-rank, derive a closed form solution for the following optimization problem:
\|A X-B\|_{F}^{2}=\operatorname{tr}\left(\mathrm{X}^{T} \mathrm{~A}^{T} A X-\mathrm{X}^{T} \mathrm{~A}^{T} B-B^{T} A X+\mathrm{B}^{T} B\right)
$$
$$
\begin{array}{l}
\frac{\partial}{\partial X}\|A X-B\|_{F}^{2}&=\frac{\partial}{\partial X} \operatorname{tr}\left(\mathrm{X}^{T} \mathrm{~A}^{T} A X-\mathrm{X}^{T} \mathrm{~A}^{T} B-B^{T} A X+\mathrm{B}^{T} B\right) \\&= A^TAX+(A^TAX-A^TB)=0-A^TB\\&=
2(A^TAX-A^TB)=0
\end{array}
$$
$$
X=(A^TA)^{-1}A^TB
$$
**Q3.** Suppose $A, C \in R^{m \times n}, b \in R^{m}$, and $\lambda$ is a constant value, derive a closed form solution for the following optimization problem:
**Q5.** Consider the diffusion equation $\frac{\partial f}{\partial t}=\frac{\partial^{2} f}{\partial x^{2}}+\frac{\partial^{2} f}{\partial y^{2}}$applying spectral methods to reduce this PDE to an ODE:____________________________________, solving this ODE in the Fourier domain and transforming the solution to the original domain using inverse Fourier transformation.
**Q6.** The variational method expresses the PDE as an integral which is to be minimized. Consider the PDE $\frac{\partial^{2} f}{\partial x^{2}}-\frac{\partial^{2} f}{\partial y^{2}}+f-a=0$, the integral to be minimized is $\min _{f} \iint $______________________________________________________________________________.
**Q8.** The Schrödinger equation and others in quantum physics involve complex number in $\mathrm{C}$. Suppose we wish to solve $A x=b$, but now $A \in \mathrm{C}^{n \times n}$ and $x, b \in$ $C^{n}$. Explain how a linear solver that takes only realvalued systems can be used to solve this equation. Hint: Write $A=A_{1}+A_{2} i$, where $A_{1}, A_{2} \in R^{n \times n} .$
A:
let $A=A_{1}+A_{2} i, b=b_{1}+b_{2} i$, and $x=x_{1}+x_{2} i$,
\left(A_{1}+A_{2} i\right)\left(x_{1}+x_{2} i\right)=b_{1}+b_{2} i
$$
$$
\left(A_{1} x_{1}-A_{2} x_{2}\right)+\left(A_{1} x_{2}+A_{2} x_{1}\right) i=b_{1}+b_{2} i
$$
$$
\begin{gathered}
A_{1} x_{1}-A_{2} x_{2}=b_{1} \\
A_{2} x_{1}+A_{1} x_{2}=b_{2} \\
{\left[\begin{array}{cc}
A_{1} & -A_{2} \\
A_{2} & A_{1}
\end{array}\right]\left[\begin{array}{l}
x_{1} \\
x_{2}
\end{array}\right]=\left[\begin{array}{l}
b_{1} \\
b_{2}
\end{array}\right]} \\
C y=d, C \in R^{2 n \times 2 n}, y, d \in R^{2 n}
\end{gathered}
$$
**Q9.** Consider the following grayscale image of the Moon as a matrix. Draw a sketch of your best guess for what the best rank-1 approximation of the image would look like.
**Q10.** Alternating least squares can be used to minimize:
$$
\min _{A, B}\|X-A B\|_{F}^{2}
$$
where $X \in R^{m \times n}, \quad A \in R^{m \times n}, \quad B \in R^{n \times n}$. In alternating least squares, you randomly initialize the first variable. You fix the first variable, and solve for second. Then you fix the second variable, and solve for the first, and you keep going until convergence. Derive a close-form solution for $A$ and $B$ separately.
&=\frac{\partial \operatorname{tr}\left(x^{\top} x-B^{\top} A^{T} x-x^{\top} A B+B^{T} A^{T} A B\right)}{d A}\\
&=\frac{-\partial [\operatorname{tr}\left(A^{\top} x B^{\top}\right)-\operatorname{tr}\left(A B x^{\top}\right)+\operatorname{tr}\left(B^{T} A^{T} A B\right)]}{\partial A}
\frac{\partial \operatorname{tr}\left(x^{\top} x-B^TA^Tx-x^{T} A B+B^T A^{\top} A B\right)}{\partial B}=2\left(A^{T} A B-A^T x\right)=0
$$
$$
B=(A^TA)^{-1}A^Tx
$$
**Q11.** Consider the equation of motion for a simple, damped,1D oscillator (a zero rest length spring in $1 \mathrm{D}$ with damping) $F(x, v)=m a=-b v-k x$,where $k$ is the spring constant, $b$ the (constant) damping coefficient,$v=x_{t}$ the velocity and $a=v_{t}=x_{t t}$ the acceleration.
(1): Show that this 2nd order ODE is equivalent to the 1st order linear system of ODEs.
$$
\left(\begin{array}{l}
x \\
v
\end{array}\right)_{t}=\left(\begin{array}{cc}
0 & 1 \\
-\frac{k}{m} & -\frac{b}{m}
\end{array}\right)\left(\begin{array}{l}
x \\
v
\end{array}\right)
$$
(2): Assume that we are using Forward Euler to solve this system numerically, show the condition for stability.
