search

jinmel / lookahead-tensorflow

Lookahead optimizer https://arxiv.org/abs/1907.08610
7 stars 3 forks source link

Adding sparse gradients #3

Open SirRob1997 opened 4 years ago

SirRob1997 commented 4 years ago

Hi,

this implementation is exactly what I needed for my purpose. It runs on tf 1.15 and extends the tf.train.Optimizer. I was looking at other implementations but failed to get how to properly properly adapt their implemented sparse gradient update into this version, any suggestions?

Best, Robin

SirRob1997 commented 4 years ago

The implementation by the authors https://github.com/michaelrzhang/lookahead/blob/master/lookahead_tensorflow.py seems to just reroute the sparse gradient method to the fast optimizer sparse gradient method which worked very poorly for me.