As aETL tool managerI wantto set execution instances to be re-tried when an error happensSo Thatit will mitigate possible orchestration failures and avoid loss of data
Additional Context
When an execution id is processed, it could be failed and the delta subset of source data may not be synchronized and missed in the next batch execution. So, with the retry option set as true, it would be re-processed every time a execution instance runs again, recovering missed data.
If in the same execution id, some processes are failed, and the parameter retry_errors is set to True, then, in the end of process it will try to run the failed processes so, if there's an error of orchestration (like misplace of dataset dependencies) it will process again the missed instances and mitigate manual/adhoc processing.
Acceptance Criteria
[ ] If in a given execution ID an instance error occur in a process and it is set to retry_errors=true, then it should re-process the failed processes with the execution_id or with same execution_id_parent.
[ ] If in a given execution ID an instance error occur in a process and it is set to retry_errors=false, no reprocess should happen.
Definition of Done
[ ] Ready to Demo in Sprint Review
[ ] Does what I have made have appropriate test coverage?
[ ] Documentation and/or scientific documentation exists and can be found
As a ETL tool manager I want to set execution instances to be re-tried when an error happens So That it will mitigate possible orchestration failures and avoid loss of data
Additional Context
Acceptance Criteria
Definition of Done