IanHG
commented
6 years ago Considerations: Build packages for each unique architecture in a heterogeneous cluster.
I think the best solution would be to have build type (e.g. based on chipset instructions used, gpu/no-gpu etc.) as separate directories under each package install directory.
This way, it is easy to only have some build types for some packages, and else just load a default. And, what do I mean by that: E.g. gcc only needs a default build type (we are probably not interested in completely optimizing the compiler for avx512, and an sse build type might be fine for all architectures), whereas another package like e.g. gromacs need special optimizations for each architecture to run optimally. In a setup like this is would be "easy" to just have a default build type for gcc that will get loaded on all nodes, and highly specialized build types for gromacs which will be loaded based on type of node.
Choosing which package to load (I think) should be done at module load time, where the module script just checks the current architecture, and then loads the most optimized build that will run on the node.
When building packages one should be able to specify on the command line which architectures to build for. Here there will be some issues with, which compiler takes which flags, but it can be solved (like I already solved some issues) by having separate packages for each compiler type that one wants to use to build the program.
Make a mechanism to load specific versions of packages dependent on architecture and instruction set on compute nodes.
This is only useable on systems where alle software is installed in a "global" location on a NFS (like grendel!).