Independent Component Analysis (ICA) is a computational method for separating a multivariate signal into additive, statistically independent non-Gaussian components. It's primarily used for blind source separation.
The ICA Algorithm:
- Centering: Subtract the mean to make the signal zero-mean
- Whitening: Transform the data to have unit variance in all directions (often using PCA)
- Estimating Independence: Find directions where components are maximally non-Gaussian
- Optimization: Iteratively adjust unmixing matrix to maximize independence
- Recover Sources: Apply unmixing matrix to mixed signals to recover independent components
Key Properties:
- Statistical Independence: Components are maximally independent from each other
- Non-Gaussianity: ICA assumes sources are non-Gaussian (works poorly with Gaussian sources)
- Ambiguities: Cannot determine the order, scale, or sign of the independent components
- Contrast Functions: Measures of non-Gaussianity like kurtosis, negentropy, etc.
Applications:
- Audio source separation (cocktail party problem)
- Artifact removal in EEG/MEG signals
- Feature extraction for image processing
- Financial data analysis
- Text and document analysis
Differences from PCA:
- PCA finds orthogonal directions of maximum variance; ICA finds independent sources
- PCA uses second-order statistics (covariance); ICA uses higher-order statistics
- PCA components are uncorrelated; ICA components are statistically independent
- PCA works with any distribution; ICA requires non-Gaussian distributions