Principal Component Analysis¶

Note

Principal Component Analysis is also available with oneAPI interfaces:

Principal Components Analysis (PCA)

Principal Component Analysis (PCA) is a method for exploratory data analysis. PCA transforms a set of observations of possibly correlated variables to a new set of uncorrelated variables, called principal components. Principal components are the directions of the largest variance, that is, the directions where the data is mostly spread out.

Because all principal components are orthogonal to each other, there is no redundant information. This is a way of replacing a group of variables with a smaller set of new variables. PCA is one of powerful techniques for dimension reduction.

Details¶

Given a set \(X = \{x_1 = (x_{11}, \ldots, x_{1p}), \ldots, x_n = (x_{n1}, \ldots, x_{np})\}\) of \(p\)-dimensional feature vectors or a \(p \times p\) correlation matrix and the number of principal components \(p_r\), the problem is to compute \(p_r\) principal directions (eigenvectors) for the data set. The library returns the transformation matrix \(T\) of size \(p_r \times p\), which contains eigenvectors in the row-major order and a vector of respective eigenvalues in descending order.

oneDAL provides two methods for running PCA:

SVD
Correlation

Eigenvectors computed by PCA are not uniquely defined due to sign ambiguity. PCA supports fast ad-hoc “sign flip” technique described in the paper [Bro07]. It modifies the signs of eigenvectors shown below:

\[\hat{T_i} = T_i \cdot sgn(\max_{1 \leq j \leq p } |{T}_{i,j}|), i=1, \ldots ,p_r\]

where \(T\)-transformation matrix is computed by PCA, \(T_i\) - \(i\)-th row in the matrix, \(j\) - column number, \(sgn\) - signum function:

\[\begin{split}sgn(x) = \begin{cases} -1, & x < 0,\\ 0, & x = 0, \\ 1, & x > 0 \end{cases}\end{split}\]

You can provide these types of input data to the PCA algorithms of the library:

Original, non-normalized data set
Normalized data set, where each feature has the zero mean and unit variance
Correlation matrix

Computation¶

The following computation modes are available:

Examples¶

Batch Processing:

pca_cor_dense_batch.cpp

Performance Considerations¶

To get the best overall performance of the PCA algorithm:

If input data is homogeneous, provide the input data and store results in homogeneous numeric tables of the same type as specified in the algorithmFPType class template parameter.
If input data is non-homogeneous, use AOS layout rather than SOA layout.

PCA computation using the correlation method involves the correlation and variance-covariance matrices algorithm. Depending on the method of this algorithm, the performance of PCA computations may vary. For sparse data sets, use the methods of this algorithm for sparse data.

Batch Processing¶

Because the PCA in the batch processing mode performs normalization for data passed as Input ID, to achieve the best performance, normalize the input data set. To inform the algorithm that the data is normalized, set the normalization flag for the input numeric table that represents your data set by calling the setNormalizationFlag() method of the NumericTableIface class.

Because the PCA with the correlation method (defaultDense) in the batch processing mode is based on the computation of the correlation matrix, to achieve the best performance, precompute the correlation matrix. To pass the precomputed correlation matrix to the algorithm, use correlation as Input ID.

Online Processing¶

PCA with the SVD method (svdDense) in the online processing mode is at least as computationally complex as in the batch processing mode and has high memory requirements for storing auxiliary data between calls to compute(). On the other hand, the online version of the PCA with the SVD method may enable you to hide the latency of reading data from a slow data source. To do this, implement load prefetching of the next data block in parallel with the compute() method for the current block.

Distributed Processing¶

PCA with the SVD method (svdDense) in the distributed processing mode requires gathering local-node \(p \times p\) numeric tables on the master node. When the amount of local-node work is small, that is, when the local-node data set is small, the network data transfer may become a bottleneck. To avoid this situation, ensure that local nodes have a sufficient amount of work. For example, distribute the input data set across a smaller number of nodes.

Product and Performance Information

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.

Notice revision #20201201

oneDAL documentation