Signal failed job step at job and order level

tangobravo62 commented 4 years ago

Currently job steps fail "silently", i. e. they are set to FAILED, but the associated job and order are not according to the state transitions defined for jobs and orders (status FAILED must only be set, when all job steps are either completed or failed).

Still from an operational point of view it is necessary to have an indicator on the higher levels that some problem is present and operator interaction may be needed.

Decision: Add a new flag hasFailedJobSteps to the Order and Job classes, which is set as soon as at least one job step fails.

Implementation note: When retrying an order, job or job step, it must be checked whether there are still failed job steps for the job and order involved, and if not the flag must be cleared again. The flag must also be cleared, when the order is reset.

tangobravo62 commented 4 years ago

Attribute hasFailedJobSteps can now be set, retrieved and changed for orders:

prosEO (PTM)> order show
---
- id: 80
  version: 5
  missionCode: "PTM"
  identifier: "L3_products_9:30-17:30"
  [...]
  hasFailedJobSteps: false

prosEO (PTM)> order update "L3_products_9:30-17:30" hasFailedJobSteps=true
(I2938) Order with identifier L3_products_9:30-17:30 updated (new version 6)

prosEO (PTM)> order show "L3_products_9:30-17:30"
---
- id: 80
  version: 6
  missionCode: "PTM"
  identifier: "L3_products_9:30-17:30"
  [...]
  hasFailedJobSteps: true

Attribute can be retrieved via CLI for job steps (modification of job steps through CLI is not possible):

prosEO (PTM)> job show -v "L2_orbits_3000-3002"
---
- id: 107
  version: 4
  orderIdentifier: "L2_orbits_3000-3002"
  jobState: "COMPLETED"
  [...]
  hasFailedJobSteps: false
  [...]

tangobravo62 commented 3 years ago

Now reported up to the GUI level:

dlr-eoc / prosEO

Signal failed job step at job and order level #83