search

acep-uaf / camio-meter-streams

0 stars 0 forks source link

Create Summary for each download run #164

Closed aloralove closed 1 month ago

aloralove commented 1 month ago

Creating a summary for each download run. What meters were accessed or failed. Events that were downloaded for each meter and then any errors.

MeterA
-status: fail/success
-errors:
   -error 200
-
...
nicolemah99 commented 1 month ago

Trying to accomplish

meters:
    - name: MeterA
      status: failure
      errors:
        - "[ERROR] SIGINT received. Exiting.... Exit code: 130"
      started_at: YYYY-MM-DD HH:II:SS
      completed_at: YYYY-MM-DD HH:II:SS
    - name: MeterB
      status: success
      events:
        - event_id: 12345
           status: success
        - event_id: 12345
           status: failure
           errors:
      warnings:
      errors:
        - "[ERROR] SIGINT received. Exiting.... Exit code: 130"
      started_at: YYYY-MM-DD HH:II:SS
      completed_at: YYYY-MM-DD HH:II:SS
nicolemah99 commented 1 month ago

I currently have it working as:

meters:
  - name: meter_1
    status: "failure"
    started_at: "2024-07-23T21:38:22+00:00"
    completed_at: "2024-07-23T21:38:25+00:00"
    events: []
    errors:
      - error_code: "230"
        error_message: 'Download failed for meter: meter_1'
  - name: meter_2
    status: "success"
    started_at: "2024-07-23T21:38:25+00:00"
    completed_at: "2024-07-23T21:46:06+00:00"
    events:
      - event_id: "10191"
        status: success
      - event_id: "10190"
        status: success
      - event_id: "10189"
        status: success
      - event_id: "10188"
        status: success
      - event_id: "10187"
        status: success
      - event_id: "10186"
        status: success
      - event_id: "10185"
        status: success
    errors: []
  - name: meter_3
    status: "failure"
    started_at: "2024-07-23T21:46:06+00:00"
    completed_at: "2024-07-23T21:49:08+00:00"
    events:
      - event_id: "10191"
        status: success
      - event_id: "10190"
        status: success
    errors:
      - error_code: "230"
        error_message: 'Download failed for meter: meter_3'
nicolemah99 commented 1 month ago

Now its working like

summary:
  started_at: "2024-07-25T21:52:01+00:00"
  completed_at: "2024-07-25T21:59:01+00:00"
  duration: "420s"
  meters_total: 4
  meters_attempted: 4
  meters_successful: 2
  meters_failed: 2
meters:
  - name: "sel00"
    status: "success"
    started_at: "2024-07-25T21:52:01+00:00"
    completed_at: "2024-07-25T21:54:00+00:00"
    duration: "119s"
    downloads:
      total: 175
      success: 175
      fail: 0
      skipped: 0
    events:
      - event_id: 10175
        status: "success"
      - event_id: 10174
        status: "success"
      - event_id: 10173
        status: "success"

...
...
    - event_id: 10144
        status: "failure"
        exit_code: 199
        message: "Failed to validate event files for event: 10144"

going to add download speed for each event