Lesson material for beginning cluster operators?

[reid-a]

This is a bit off-the-wall, but there was recent e-mail traffic on the OMPI Users mailing list from an inexperienced HPC cluster operator/user asking how to optimally build OpenMPI. The (correct) response was "it depends", but also included links to some "getting started" resources.

The broader idea that this triggered for me was whether cluster set-up lessons are something we should consider as a useful contribution we could make.

Pro:

• Clusters are getting more common and easier to buy, but expertise is not expanding, so this is useful
• It's kind of adjacent to what we are thinking about for task-specific or ephemeral cloud HPC resources
• Small academic cluster ops is one of those classic grad-student tasks, with the attendant loss of institutional memory when the operators leave. Carpentries is in part about helping retain expertise and utility over personnel changes.

Con:

• We have lots to do already, it would dilute our focus
• This is a hard problem, developing material around "it depends" is a fool's errand
• The new user audience we are already addressing is larger and more important than the audience of new operators.

I think I lean slightly "con", personally, mostly for focus reasons, but if additional partticipants were forthcoming, I feel like this is in scope.

[tkphd]

I think this would tie in nicely with the MagicCastle/cloud resource/interest-sans-equipment demographic in our community. I'm actually in favor of this, esp. if we can focus the first draft on a specific hardware or cloud platform (RaspberryPi? AWS? Smaller, cooler provider?).