search

dessn / Pippin

Pipeline for photometric SN analysis
MIT License
9 stars 10 forks source link

Pippin should allow manual update of ALL.DONE #118

Closed RickKessler closed 1 year ago

RickKessler commented 1 year ago

If a pippin task fails, user should be able to manually fix it and set SUCCESS in ALL.DONE. Example case is LC fit on ~10^5 jobs, and only 1 job fails.

OmegaLambda1998 commented 1 year ago

Current workflow is:

  1. Create hash.txt on task creation
  2. Run task
  3. If task failed (not crash), remove hash so that task is guaranteed to rerun next time.
  4. Set ALL.DONE to FAIL
  5. If task crashed (not failed), hash is not removed, and ALL.DONE is not guaranteed to be set I've made it so that whenever the hash is removed, it is first copied over to hash_old.txt so that if ALL.DONE is set to SUCCESS, you can simply mv hash_old.txt hash.txt to force Pippin into accepting the change. Note that if the Pippin job has changed, the hash will still be different and pippin will want to rerun. In that case you can just use -i N where N is the task number, to ignore hash checks and not rerun up to and included the Nth task