jameschapman19 / cca_zoo

Canonical Correlation Analysis Zoo: A collection of Regularized, Deep Learning based, Kernel, and Probabilistic methods in a scikit-learn style framework
https://cca-zoo.readthedocs.io/en/latest/
MIT License
190 stars 40 forks source link

SVD Not Converging #193

Open BogyeomKim opened 5 months ago

BogyeomKim commented 5 months ago

Hello. This is Bogyeom Kim.

I've been using the SCCA_PMD function from the cca_zoo package for Sparse Canonical Correlation Analysis on my datasets. However, I've encountered a recurring error related to SVD not converging: i = 1, SVD did not converge.

I also encountered a recurring error following below: /home/connectome/kimbo/.conda/envs/bct_cca/lib/python3.8/site-packages/cca_zoo/models/_rcca.py:123: RuntimeWarning: invalid value encountered in sqrt / np.sqrt(eigvals[idx]) /home/connectome/kimbo/.conda/envs/bct_cca/lib/python3.8/site-packages/cca_zoo/models/_iterative/_base.py:154: UserWarning: Some scores are nan. Usually regularisation is too high. warnings.warn( /home/connectome/kimbo/.conda/envs/bct_cca/lib/python3.8/site-packages/cca_zoo/models/_iterative/_base.py:105: RuntimeWarning: invalid value encountered in divide return residual - np.outer(score, score) @ residual / np.dot(score, score) /home/connectome/kimbo/.conda/envs/bct_cca/lib/python3.8/site-packages/cca_zoo/models/_iterative/_base.py:92: UserWarning: Inner loop 31 not converged. Increase number of iterations. warnings.warn( /home/connectome/kimbo/.conda/envs/bct_cca/lib/python3.8/site-packages/numpy/lib/function_base.py:2854: RuntimeWarning: invalid value encountered in divide c /= stddev[:, None] /home/connectome/kimbo/.conda/envs/bct_cca/lib/python3.8/site-packages/numpy/lib/function_base.py:2855: RuntimeWarning: invalid value encountered in divide c /= stddev[None, :]

Here's a brief overview of my process:

Could you please shed some light on:

  1. The specific part of the SCCA_PMD process where SVD is applied?
  2. Any recommendations or steps I might take to resolve the SVD convergence issue within the context of SCCA_PMD?

Any insights or suggestions you could provide would be greatly appreciated. I've ensured the data scaling was appropriately applied and hoped the penalty parameters from the grid search would suffice, but it seems there may be other factors at play affecting SVD convergence.

Thank you for your time and for developing such a useful tool.

Best regards, Bogyeom Kim

jameschapman19 commented 4 months ago

Hi,

I'm just looking into this. Reproducible script would be great!

jameschapman19 commented 4 months ago

So one thing to say is in more recent versions I have changed SCCA_PMD to SPLS. Later versions should in general be much better/fewer bugs.

I'd need to see where the SVD error is coming from a printout say to even have a chance of working this one out.

Basically I'd try using a more recent version