8.2 James-Stein estimation

It may be worthwile to perform a simple simulation to check the potential advantage of using non standard estimators for the covariance matrix, the key quantity in PCA, in the case of large dimensionality and small number of samples.

Having generated our data sample we compute the covoriance matrix using the usual sample estimator and the shrinkage estimator:

and proceed to the computation of the eigenvalues:

As reported in Figure 8.4, the advatange of shrinkage estimators over the standard ones can be dramatic.

Figure 8.4: A good estimate of the covariance matrix is key to perform meaningful PCA. While the MLE sample estimate is commonly employed, it is far from optimal in the high dimensionality/few samples case.