search

clusterinthecloud / support

If you need help with Cluster in the Cloud, this is the right place
2 stars 0 forks source link

Choice of headnode instance #13

Open tomdeakin opened 3 years ago

tomdeakin commented 3 years ago

How cross platform are the headnode instances? It's often very painful if the mgmt architecture doesn't match the compute node architectures. For instance, on AWS the mgmt node is AMD but we might be launching Arm-based compute nodes. This means I have to compile inside a running job, or deal with installing and using cross-compilers.

milliams commented 3 years ago

Currently the arch of the head node is fixed to reduce the maintenance burden on us. Our recommendation (and how Chris did it before) is to do the compilation in a running job.

Probably the easiest method is to start an interactive job (for example the srun method here) and do the compilation in there.

Since the head node is usually very small (1 or 2 cores and only a few gigs of RAM) it is not usually suitable for large compilations.

tomdeakin commented 3 years ago

Makes sense, thanks. Might make a great summer project for someone to add support for this 😄