Closed chryswoods closed 2 years ago
I'm planning on in the future fetching all shapes dynamically from the API so this will have to change then. It's part of a larger idea to allow more flexible requests across the board. This acts as a good case study of the sort of requirements it will have.
Thanks - I agree that a more dynamic approach would be better :-). I think, increasingly, we will see that the concept of a "shape" will disappear, and instead we will be requesting instances with specified processor architecture, number of cores, amount of RAM, amount of disk, speed of disk, number / type of GPUs, type of network, etc. It will then be the cloud API in the back end that will resolve this to a specific node type.
It is an open question who we should present this to the user, i.e. do we still create a set of fixed (and named) compute node shapes (e.g. VM.Standard.E4.Flex.16.512) or is every node generic, and we pass the GRES options from slurm (translated) straight through to the cloud API?
(and I forgot - most importantly - specifying the availability of that instance, i.e. interruptible, burstable etc)
Done this by adding the number of CPUs and amount of memory to the node name, e.g.
VM.Standard.E4.Flex.64.1024
would be a 64-core, 1024 GB VM.Standard.E4.Flex node.
Tested this and it all works :-)
(need to have the appropriate shapes.yaml from the terraform repo)