Closed mfschubert closed 1 month ago
I'm starting a few tests on my problems with that change, thanks. It'll take several hours as I'm working with a rather large number of Fourier terms in the decomposition.
So, I have not done extensive testing (and doing so would likely require me to retune all my optimizer hyperparameters), but from the light testing I've done, this new scheme (including the latest commit) does seem to behave quite well on my setup, possibly even better than what I was getting before by setting $\epsilon = 10^{-20}$ :+1: .
Motivated by https://github.com/facebookresearch/fmmax/issues/105, a new regularization scheme.
jnp.linalg.eigh
, which is differentiable "out of the box". The tests include matrices with shifted and scaled eigenvalues, which would fail with the previous regularization scheme.@elinorbgr could you test this? @smartalecH good to get your eyes on this also