search

interTwin-eu / itwinai

Advanced AI workflows for digital twins applications in science.
https://itwinai.readthedocs.io
MIT License
15 stars 5 forks source link

Switch to non-patched Horovod installation #210

Closed matbun closed 1 month ago

matbun commented 1 month ago

As of now, when building the torch environment we had to force the constraint torch<2.2.0 to be able to install horovod.

Until before https://github.com/horovod/horovod/pull/3998 was merged, the minimum required C++ version for compiling PyTorch 2.1 was C++17 (https://github.com/pytorch/pytorch/pull/100557) while Horovod required C++11, preventing the Horovod wheel to build correctly. Therefore, we used to install horovod from the "patched" fork this way:

pip install --no-cache-dir git+https://github.com/thomas-bouvier/horovod.git@compile-cpp17

However, since https://github.com/horovod/horovod/pull/3998 is now merged, it would be better to install from main until a new version is released (e.g., Horovod 0.28.2):

pip install --no-cache-dir git+https://github.com/horovod/horovod.git

Moreover, it would be beneficial to test the Horovod installation with newer versions of torch (e.g., 2.3). Related issue: #199

jarlsondre commented 1 month ago

Tested running version 2.4.0 in the generic_script_macos branch and it works both locally and on JSC. Also updated Horovod and that worked too.