search

fredhallgren / inkpca

Incremental kernel PCA
Apache License 2.0
28 stars 8 forks source link

How to build KPCA components and perform inverse transform? #8

Open pengmy001 opened 2 years ago

pengmy001 commented 2 years ago

Thank you for your wonderful work and algorithm! I found it really helpful. Now I am trying to apply inkpca to a large image dataset. However, I found that in the 'experiments.py', there are no results for kpca final components and its inverse transformation results, which would really help my work. Is there any possiblity that this algorithm can include calculating components and inverse transformation?

fredhallgren commented 2 years ago

Thank you for your interest in the package!

The inverse transformations are not necessarily straightforward or possible to compute, depending on the kernel. Can calculate the representations of the data points in terms of the principal components (the projection coefficients of each data points with respect to the principal components, also known as principal scores https://en.wikipedia.org/wiki/Principal_component_analysis). This would be quite easy to do, just need to scale the eigendecomposition of kernel matrix by the eigenvalues, if I have some time the coming days I can do this.

If you don't need the incremental computation I'm doing this already in my other package at [https://github.com/fredhallgren/nystrompca/blob/develop/nystrompca/algorithms/kernel_PCA.py]. Can look at bottom of page 10 at [https://arxiv.org/abs/2109.05578] for the expression to calculate.

There is a slightly faster algorithm for incremental kernel PCA in a paper coming out soon by some other people, I'm considering implementing this algorithm in this package (their current code is not in Python).

Kind regards

Fred

London38 commented 2 years ago

Thanks @fredhallgren for your amazing work,

Here is another algorithm would ya like to read, It's also about a a faster incremental kernel PCA. Ty ;)

fredhallgren commented 2 years ago

Interesting, do you have a link?

pengmy001 commented 2 years ago

Thank you for your interest in the package!

The inverse transformations are not necessarily straightforward or possible to compute, depending on the kernel. Can calculate the representations of the data points in terms of the principal components (the projection coefficients of each data points with respect to the principal components, also known as principal scores https://en.wikipedia.org/wiki/Principal_component_analysis). This would be quite easy to do, just need to scale the eigendecomposition of kernel matrix by the eigenvalues, if I have some time the coming days I can do this.

If you don't need the incremental computation I'm doing this already in my other package at [https://github.com/fredhallgren/nystrompca/blob/develop/nystrompca/algorithms/kernel_PCA.py]. Can look at bottom of page 10 at [https://arxiv.org/abs/2109.05578] for the expression to calculate.

There is a slightly faster algorithm for incremental kernel PCA in a paper coming out soon by some other people, I'm considering implementing this algorithm in this package (their current code is not in Python).

Kind regards

Fred

Is it possible that you can add data reconstruction(inverse inkpca) in your repository? That would really help people like me who are struggling to acheive this on a really large dataset. Thanks!