search

lucidrains / lion-pytorch

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch
MIT License
2.04k stars 50 forks source link

AttributeError: module 'trinton.language' has no attribute 'constexpr' #5

Closed mlw214 closed 1 year ago

mlw214 commented 1 year ago

Tried enabling Triton and ran into the above issue. I installed the latest version of Triton on PyPi (1.1.1). Is there a particular version this library is compatible with?

lucidrains commented 1 year ago

@mlw214 oh, i think that version may be dated

could you try pip install triton -U --pre?

mlw214 commented 1 year ago

Yep, that fixed it! Thanks for the quick response. May be prudent to call that out in the README that you need the a prerelease of Triton.

lucidrains commented 1 year ago

@mlw214 yup, done here, happy training! do leave your experience with the new optimizer in the discussions

lucidrains commented 1 year ago

@mlw214 how did it go?

mlw214 commented 1 year ago

@lucidrains currently working out a good LR range that doesn't cause the loss to NaN. After a few tries I got one run to finish up without problems. To add some context I'm actually throwing this at an object detection problem where I've traditionally used MADGRAD as my optimizer of choice. I'm hoping to have some full experiments done over the next week comparing the two (I'll also try to throw AdamW in the mix, but I've found it generally performs worse than MADGRAD). I'll post to the discussion thread the results, along with more details on the architecture, setup, etc.

lucidrains commented 1 year ago

@mlw214 looking forward to it!