First, great project. Seeing good improvements over straight Adam. Have you seen the Sharpness-Aware Minimization for Efficiently Improving Generalization paper? I'm curious if you have any thoughts on how to integrate Ranger21 with a SAM implementation ( https://github.com/davda54/sam for example )
First, great project. Seeing good improvements over straight Adam. Have you seen the Sharpness-Aware Minimization for Efficiently Improving Generalization paper? I'm curious if you have any thoughts on how to integrate Ranger21 with a SAM implementation ( https://github.com/davda54/sam for example )
Thanks!