materialsproject / fireworks

The Fireworks Workflow Management Repo.
https://materialsproject.github.io/fireworks
Other
361 stars 185 forks source link

Handling of locked fireworks when launching many concurrent jobs could be improved #490

Open Andrew-S-Rosen opened 2 years ago

Andrew-S-Rosen commented 2 years ago

This issue was discussed at length by @rkingsbury on the forum here. To briefly reiterate the main point, when lots of fireworks are launched in parallel (e.g. a jobpack) and then many are cancelled or aborted, the user is often left with one or more locked fireworks that are: 1) very time-consuming to identify, and 2) very time-consuming to unlock. I run into this problem probably once a month, which is why I've decided to open an "official" issue about it. Ideally, there should be a new CLI argument that lets you unlock all your fireworks. Of course, using this while jobs are actively running is a bit dangerous, but I think having the utility to use when it's safe to do so would be a huge help.