PCA理论简述

文章翻译自pca_brief_introduction.pdf的第二节，原始链接是这里。

今天有需要查一查PCA是个什么意思，看了几个资料，这个讲的最简单的，易懂。<nobr style="border:0px; padding:0px; margin:0px; max-width:none; max-height:none; vertical-align:0px"><span class="math" id="MathJax-Span-1" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:16px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:relative; width:204px; height:0px"><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:absolute; top:-34px; left:0px"><span class="mrow" id="MathJax-Span-2" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="texatom" id="MathJax-Span-3" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-4" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-5" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">吐</span></span></span></span><span class="texatom" id="MathJax-Span-6" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-7" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-8" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">槽</span></span></span></span><span class="texatom" id="MathJax-Span-9" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-10" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-11" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">下</span></span></span></span><span class="texatom" id="MathJax-Span-12" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-13" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-14" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">很</span></span></span></span><span class="texatom" id="MathJax-Span-15" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-16" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-17" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">多</span></span></span></span><span class="texatom" id="MathJax-Span-18" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-19" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-20" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">中</span></span></span></span><span class="texatom" id="MathJax-Span-21" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-22" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-23" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">文</span></span></span></span><span class="texatom" id="MathJax-Span-24" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-25" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-26" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">博</span></span></span></span><span class="texatom" id="MathJax-Span-27" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-28" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-29" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">客</span></span></span></span><span class="texatom" id="MathJax-Span-30" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-31" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-32" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">，</span></span></span></span><span class="texatom" id="MathJax-Span-33" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-34" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-35" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">说</span></span></span></span><span class="texatom" id="MathJax-Span-36" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-37" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-38" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">的</span></span></span></span><span class="texatom" id="MathJax-Span-39" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-40" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-41" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">完</span></span></span></span><span class="texatom" id="MathJax-Span-42" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-43" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-44" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">全</span></span></span></span><span class="texatom" id="MathJax-Span-45" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-46" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-47" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">不</span></span></span></span><span class="texatom" id="MathJax-Span-48" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-49" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-50" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">明</span></span></span></span><span class="texatom" id="MathJax-Span-51" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-52" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-53" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">白</span></span></span></span></span><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; width:0px; height:34px"></span></span></span><span style="border-width:0px; border-left-style:solid; outline:0px; vertical-align:-4.2px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; overflow:hidden; width:0px; height:18.2px"></span></span></nobr>。翻译下理论简述。

---

PCA<nobr style="border:0px; padding:0px; margin:0px; max-width:none; max-height:none; vertical-align:0px"><span class="math" id="MathJax-Span-54" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:16px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:relative; width:72px; height:0px"><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:absolute; top:-34px; left:0px"><span class="mrow" id="MathJax-Span-55" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="texatom" id="MathJax-Span-56" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-57" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-58" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">主</span></span></span></span><span class="texatom" id="MathJax-Span-59" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-60" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-61" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">要</span></span></span></span><span class="texatom" id="MathJax-Span-62" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-63" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-64" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">成</span></span></span></span><span class="texatom" id="MathJax-Span-65" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-66" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-67" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">分</span></span></span></span><span class="texatom" id="MathJax-Span-68" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-69" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-70" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">分</span></span></span></span><span class="texatom" id="MathJax-Span-71" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mrow" id="MathJax-Span-72" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mo" id="MathJax-Span-73" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:static; font-family:STIXGeneral,'Arial Unicode MS',serif">析</span></span></span></span></span><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; width:0px; height:34px"></span></span></span><span style="border-width:0px; border-left-style:solid; outline:0px; vertical-align:-4.2px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; overflow:hidden; width:0px; height:18.2px"></span></span></nobr>是使用统计学的技术对高维数据进行模式查找的有用方法。这个方法，首先我们尝试着去寻找给定数据的方差，然后将观察到的模式用更小维度的数据表示。例如给定一个矩阵A，他的协方差矩阵可以使用<nobr style="border:0px; padding:0px; margin:0px; max-width:none; max-height:none; vertical-align:0px"><span class="math" id="MathJax-Span-74" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; font-size:16px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:relative; width:35px; height:0px"><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:absolute; top:-34px; left:0px"><span class="mrow" id="MathJax-Span-75" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span class="mi" id="MathJax-Span-76" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static; font-family:MathJax_Math; font-style:italic">A</span><span class="msubsup" id="MathJax-Span-77" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static"><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:relative; width:22.2px; height:0px"><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:absolute; top:-34px; left:0px"><span class="mi" id="MathJax-Span-78" style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static; font-family:MathJax_Math; font-style:italic">A</span><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; width:0px; height:34px"></span></span><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; position:absolute; top:-41.2px; left:12px"><span class="mi" id="MathJax-Span-79" style="border:0px; outline:0px; font-size:12px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline; position:static; font-family:MathJax_Math; font-style:italic">T<span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; overflow:hidden; height:1px; width:1.4px"></span></span><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; width:0px; height:34px"></span></span></span></span></span><span style="border:0px; outline:0px; vertical-align:0px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; width:0px; height:34px"></span></span></span><span style="border-width:0px; border-left-style:solid; outline:0px; vertical-align:-1px; background-color:transparent; margin:0px; padding:0px; display:inline-block; position:static; overflow:hidden; width:0px; height:16.9px"></span></span></nobr>来求出。我们知道，对于任何一个矩阵，他的空间中的特征向量仅仅在同样的方向上缩放。也就是说，如果我们能够找到协方差矩阵，我们就能够知道这个矩阵在特定方向上数据的大小。与特征向量对应的特征值可以让我们知道这个矩阵在这个方向上的数量级。也就是说，原始图像中的任何一个向量可以被特征向量的线性组合来表示。

如果我们忽略一些方向上的小量级的数据，在使用特征向量重新构建这个矩阵。这个新的矩阵和原始矩阵会非常近似，但是数据的维度会明显降低。这些剩下的特征向量，叫做这个矩阵的主要成分。

----

欢迎关注我的新浪微博@TJUReyoung, 欢迎访问我的博客。