Add tutorial for using Pyrokinetics on HPC systems

[LiamPattinson]

A while back, I recall talking about a potential feature that could automatically dispatch gyrokinetics jobs to a remote HPC system (although I don't think an issue was raised for this). Having looked into it a bit, I think it would be very difficult to design a generic tool for doing this, so instead I've written a short guide on how it could be achieved for a simple toy problem. It first explains how to install Pyrokinetics on an HPC machine, and then gives a very rough overview of two tools that can be used to automate Pyrokinetics runs:

• QCG-PilotJob: A job scheduler that runs within a single Slurm job. Allows you to queue up a large number of jobs and manages resource allocation for you.
• HPC Rocket: A tool for dispatching jobs to remote HPC machines without the need to SSH in. Mostly intended as a CI tool, but it can used as a regular command line tool too.

Hopefully this might be useful to somebody! If not, I've at least had some fun playing around with new tools.

[LiamPattinson]

Thanks for the review! I've fixed up all of the inconsistencies, added some details on srun usage, and added some info on using SSH keys instead of supplying a password to HPC Rocket (I think a password might make more sense when you're running it as a CI job, as you can just save your password to a GitHub secret, but you're right that SSH keys are preferable in all other cases).

[LiamPattinson]

I've added a few lines about unlocking private key passwords beforehand. I'd actually been having issues with my keychain for a while, and this seems to have fixed it, so cheers for the prompt!

[bpatel2107]

This looks fantastic! @whornsby82 @lorenzozanisi FYI may be of interest

Is it worth including info on how to pull the output data from the HPC cluster? I think that would close the whole loop though it is a bit out of the scope.

[LiamPattinson]

There's a bit in the 'Simple usage' section about scp-ing output files back, but I guess I never explained how it works for HPC Rocket. I've added a small section to the end that explains how to ensure HPC Rocket pulls files back to you after the batch job has finished.