We add the Report model that tracks the progress of each pipeline run on a single section of an Exposure.
The report first and foremost tracks the start and (optional) end time of the pipeline.
If the pipeline didn't finish (because it is still running or hanging) it will have a Null end time.
If the pipeline had an exception, the end time should also be recorded.
There is also room to track any error that stopped the pipeline, warnings that come up, and the memory/runtime of each step.
We also use bitflags to record which processing steps were done, which pipeline products were already instantiated when the pipeline was finished, and which were successfully saved to disk/database (this one is a little tricky!).
To get this to work we also need to change how pipeline objects handle errors and make sure they always return a datastore with appended warnings and errors, so they can be saved to the report.
We add the Report model that tracks the progress of each pipeline run on a single section of an Exposure.
The report first and foremost tracks the start and (optional) end time of the pipeline. If the pipeline didn't finish (because it is still running or hanging) it will have a Null end time. If the pipeline had an exception, the end time should also be recorded.
There is also room to track any error that stopped the pipeline, warnings that come up, and the memory/runtime of each step.
We also use bitflags to record which processing steps were done, which pipeline products were already instantiated when the pipeline was finished, and which were successfully saved to disk/database (this one is a little tricky!).
To get this to work we also need to change how pipeline objects handle errors and make sure they always return a datastore with appended warnings and errors, so they can be saved to the report.