search

camall3n / onager

Lightweight python library for launching experiments and tuning hyperparameters, either locally or on a cluster
MIT License
20 stars 4 forks source link

Additional arguments to grid engine are incompatible with '-q' #28

Closed camall3n closed 1 year ago

camall3n commented 4 years ago

Currently the 'launch' subcommand passes additional unknown arguments to the backend. But '-q' is reserved for quiet-mode, so gridengine args are incompatible.

camall3n commented 4 years ago

For example, adding -q "short.q@**smblade**" results in:

onager launch: error: argument -q/--quiet: ignored explicit argument ' short.q@**smblade**'
camall3n commented 1 year ago

As for the reason the --quiet option was even in the launch subcommand, it's because previously when you would launch a large number of array tasks, it wouldn't condense them into a reasonable string like 1-1000. Instead it would produce an obnoxiously long comma-separated list like 1,2,3,...,1000. So the -q or --quiet option was to make that less obnoxious.

But it doesn't do that anymore. The slurm backend can handle condensed tasklists like 1-3,7-9, and on gridengine, it will simply send a job for each contiguous range: i.e. 1-3 and 7-9.

So, long story short, the -q argument is never useful anymore, and we can simply remove it!

camall3n commented 1 year ago

Wait, but what about the local backend you say?

Well, there might still be a use-case for -q there, but it's unclear why you wouldn't want to see the output in order to track the progress of the jobs, since the command doesn't return control until it finishes all the jobs anyway.

Remove!