sail-sg / Adan

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
Apache License 2.0
756 stars 64 forks source link

Allow unconditional CUDA build #30

Closed janEbert closed 1 year ago

janEbert commented 1 year ago

Sometimes I have to compile packages on a non-CUDA machine and later use that compilation on a CUDA machine. This PR unconditionally builds the CUDA extension when FORCE_CUDA=1 is set.

Also I tried to give helpful warnings when users stumble upon the same error; also, we error out early if the user does not have a GPU but tries using fused Adan.

Finally, I took the opportunity to also include an increase in the version string here. That way, when people pip install -U ...Adan.git, they get fused Adan without having to force-reinstall the package (otherwise pip does not see that there's a new feature available).