search

AGI-Arena / MARS

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models
https://github.com/AGI-Arena/MARS
Apache License 2.0
200 stars 33 forks source link

got a warning, is it harmless? #1

Open saurabh-kataria opened 1 week ago

saurabh-kataria commented 1 week ago

Thanks for your work. MARS is working wonderfully well on a range of architectures in my personal projects. I just wanted to make sure I am not missing out on more benefits 😁

./MARS/MARS/optimizers/mars.py:23: UserWarning: This overload of add is deprecated: add(Number alpha, Tensor other) Consider using one of the following signatures instead: add_(Tensor other, *, Number alpha = 1) (Triggered internally at /opt/conda/conda-bld/pytorch_1729647348947/work/torch/csrc/utils/python_arg_parser.cpp:1642.) expavg.mul(beta1).add_(1. - beta1, c_t)

lauyikfung commented 1 week ago

Thank you for your notice. It is just because we used the old version of usage of "add_" function, and the warning does not affect the performance. To avoid the warning, we have updated the optimizer file. :)