AttributeError: module 'torch' has no attribute '_foreach_lerp_'

facebookresearch / schedule_free

Schedule-Free Optimization in PyTorch

Apache License 2.0

1.54k stars 49 forks source link

AttributeError: module 'torch' has no attribute '_foreach_lerp_' #26

Closed patriksimurka closed 1 month ago

patriksimurka commented 1 month ago

Hi, nice work! I'm trying to use this in my project, but I'm getting this:

    optimizer.step()
  File "/home/patriksimurka/miniconda3/envs/equiformer_v2/lib/python3.8/site-packages/torch/optim/optimizer.py", line 140, in wrapper
    out = func(*args, **kwargs)
  File "/home/patriksimurka/miniconda3/envs/equiformer_v2/lib/python3.8/site-packages/schedulefree/adamw_schedulefree.py", line 164, in step
    torch._foreach_lerp_(y, z, weight=ckp1)
AttributeError: module 'torch' has no attribute '_foreach_lerp_'

adefazio commented 1 month ago

Thanks for pointing out this bug! It will run if you set foreach=False. Older versions of pytorch don't support the faster foreach implementation. I need to add some code to detect this and fallback to the non-foreach version.