AI4OPT / OPFGenerator

Instance generator for OPF problems
MIT License
2 stars 3 forks source link

Provide option to skip sysimage #41

Closed mtanneau closed 4 months ago

mtanneau commented 4 months ago

The current automated slurm workflow consists of 4 steps, each one in its own job and each job depending on the previous one:

  1. sysimage
  2. ref
  3. sampler
  4. extract

We currently do not support running multiple systems (eg 118_ieee, 300_ieee, 1354_pegase...) at the same time. Therefore, a user needs to run the above workflow multiple times, one for each system. This can create conflicts for the sysimage job, since different jobs would attempt to save a sysimage at the same location (currently app/julia.so).

In addition, if a user wants to skip the sysimage step (e.g. they have already compiled one), they need to manually modify the submit.sh script in order to remove the --dependency that the ref job has on the sysimage job.

Supporting multiple systems in the same config would be more work than we can afford at the moment. Instead, I would suggest we separate the sysimage creation from the other 3 jobs. This would add one manual step (creating the sysimage) before one can submit the ref/sampler/extract series of jobs.

klamike commented 4 months ago

Agreed. I always did this manually. I can make the submit.sh script interactive and ask the user if they want to skip the sysimage job.