Invalidate checkpoint files when config dict changes

LSSTDESC / imSim

GalSim based Rubin Observatory image simulation package

https://lsstdesc.org/imSim

BSD 3-Clause "New" or "Revised" License

36 stars 15 forks source link

Invalidate checkpoint files when config dict changes #382

Open rmjarvis opened 1 year ago

rmjarvis commented 1 year ago

The docs have this warning currenty:

.. warning::

    Be careful to manually delete any check-point files if you have made any changes to to the configuration between runs.  Currently, *imSim* only checks if a file for a individual sensor already exists.

I think we can fix this by including a hash of the base dict in the checkpoint "name" to automatically start over when the config file changes.

cwwalter commented 1 year ago

This would be a good idea. I just got bitten by this in a way I was surprised by.

I was making changes to stamp.py. There were some mistakes but when I ran the program, it was fine and they didn't get caught. That's because the stamp/output was being skipped all together due to the presense of the checkpoints. It was caught by the tests in the CI. It took me quite a while to notice what was going on.