nasa / opera-sds-ops

Apache License 2.0
4 stars 2 forks source link

[New Feature]: Identify jobs that failed in phase of pipeline and re-submit processing from that phase only #24

Open riverma opened 1 year ago

riverma commented 1 year ago

Checked for duplicates

Yes - I've already checked

Alternatives considered

Yes - and alternatives don't suffice

Related problems

@LalaP mentioned there is currently no OPS procedure to clean the system and relaunch jobs at specific phases of the R1 processing pipeline. The result of this is that OPS, for example during failed hand-shakes with the DAAC, has to clean the entire system out of specific granules, and re-download and go through all processing phases in order to deal with a failed CNM receive message for specific granules.

Describe the feature request

@LalaP mentioned the need for OPS to be able to relaunch jobs at any phase of the processing pipeline, i.e. download, ingest, PGE execution, CNM notify, CNM receive.

Things needed to resolve this ticket: