Customization options for esi_cluster_setup

esi-neuroscience / acme

Asynchronous Computing Made ESI

https://esi-acme.readthedocs.io/en/latest/

BSD 3-Clause "New" or "Revised" License

11 stars 2 forks source link

Customization options for esi_cluster_setup #31

Closed KatharineShapcott closed 3 years ago

KatharineShapcott commented 3 years ago

Hi Stefan, I would like to be able to give my own job_extra commands. I'm imaging a default input that's an empty list that you could fill with your custom options: job_extra=[] But then you'd need to do a slightly ugly append thing:

esi_cluster_setup(job_extra=["--nodelist=esi-svhpc46"])
job_extra.append("--output={}".format(out_files)")
cluster = SLURMCluster(job_extra=job_extra)

What do you think? I could implement it and test it like this if you agree. Best, Katharine

pantaray commented 3 years ago

Hi Katharine! Great idea! I always wanted to provide an option to extend job_extra in esi_cluster_setup but never got around to actually do it. I think starting with an empty list and then simply appending to it is perfectly fine imho. If you have bandwidth to implement and test it, I'd really appreciate it! I'm currently pretty booked with setting up a new (and improved) JupyterHub instance so I'll probably won't be able to focus on ACME for the next week or so (cf. #30) - sorry! As soon as that is off my plate, I'll start working on it!

All the best, Stefan