facebookresearch / fmmax

Fourier modal method with Jax
MIT License
95 stars 9 forks source link

New eigenvector gradient regularization scheme #125

Closed mfschubert closed 1 month ago

mfschubert commented 2 months ago

Motivated by https://github.com/facebookresearch/fmmax/issues/105, a new regularization scheme.

@elinorbgr could you test this? @smartalecH good to get your eyes on this also

elinorbgr commented 1 month ago

I'm starting a few tests on my problems with that change, thanks. It'll take several hours as I'm working with a rather large number of Fourier terms in the decomposition.

elinorbgr commented 1 month ago

So, I have not done extensive testing (and doing so would likely require me to retune all my optimizer hyperparameters), but from the light testing I've done, this new scheme (including the latest commit) does seem to behave quite well on my setup, possibly even better than what I was getting before by setting $\epsilon = 10^{-20}$ :+1: .