search

gallantlab / pyrcca

Regularized kernel canonical correlation analysis in Python
Other
242 stars 73 forks source link

Weights all zero #26

Open nienkevanunen opened 3 years ago

nienkevanunen commented 3 years ago

Hello,

I've been using this package for a short while now to try to do cross modal retrieval. However, for some reason my weights are always zero or near zero (e.g. -0.001), no matter what combination of data I use, or what combination of components and regularization. And this is not the case when I run the example notebook. Additionally, when I use other cca packages, it is also not happening. What could be the cause of this? Surely the predictions can't be that great if the weights that it is using are zero...

image

TomDLT commented 3 years ago

Hi, have you tried rescaling your input data, for example dividing the arrays with their standard deviation ?

nienkevanunen commented 3 years ago

Hi, have you tried rescaling your input data, for example dividing the arrays with their standard deviation ?

I hadn't, but that has certainly changed things. Thank you for the tip!

nienkevanunen commented 3 years ago

Though, as soon as I add any regularization (the example above used 0), all the weights are back to 0.

TomDLT commented 3 years ago

What values did you use for regularization ? Adequate regularization can vary by several orders of magnitude depending on the problem. See for example the default range of regularization in CCACrossValidate, from 1e-3 to 1e1: https://github.com/gallantlab/pyrcca/blob/bc60f25a8e395baf495dd0f7234a68490ed16a22/rcca/rcca.py#L160