JuliaLang / juliaup

Julia installer and version multiplexer
MIT License
931 stars 81 forks source link

Support for heterogenous clusters with shared file systems? #940

Closed paulmelis closed 4 weeks ago

paulmelis commented 4 weeks ago

I was in the process of adding juliaup to EasyBuild (https://github.com/easybuilders/easybuild-easyconfigs/pull/20318) and one of the questions that came up was how it handled heterogeneous clusters/HPC systems. Say where login nodes are regular x64, but some compute nodes are aarch64 (say NVIDIA Grace Hopper).

Juliaup itself seems to detect and handle the architecture it's running on fine, and will download whatever binary is needed (as long as Julia provides it for the specific architecture). But the snag I ran into is that the configuration file(s) used seems to be tied to a specific architecture:

paulm@j21n14:~/software-arm64/juliaup/bin$ ./julia
The latest version of Julia in the `release` channel is 1.10.3+0.aarch64.linux.gnu. You currently have `1.10.3+0.x64.linux.gnu` installed. Run:

  juliaup update

in your terminal shell to install Julia 1.10.3+0.aarch64.linux.gnu and update the `release` channel to that version.

Here, I installed juliaup on the x64 login node of a cluster, moved it out of ~/.juliaup, and then on an aarch64 compute node installed juliaup and moved it some ~/software-arm64 (in order to get separate sets of juliaup binaries). But then using the aarch64 version of juliaup on the compute node wants to overwrite the downloaded x64 julia binaries as managed by the x64 version of juliaup.

davidanthoff commented 4 weeks ago

Yes, that doesn't work... I'll close this as a duplicate of https://github.com/JuliaLang/juliaup/issues/455, ok?