search

catalyst-cooperative / pudl-archiver

A tool for capuring snapshots of public data sources and archiving them on Zenodo for programmatic use.
MIT License
4 stars 1 forks source link

Publish September 1st 2024 archives #412

Closed github-actions[bot] closed 2 months ago

github-actions[bot] commented 2 months ago

Summary of results:

See the job run logs and results here.

Review and publish archives

For each of the following archives, find the run status in the Github archiver run. If validation tests pass, manually review the archive and publish. If no changes detected, delete the draft. If changes are detected, manually review the archive following the guidelines in step 3 of README.md, then publish the new version. Then check the box here to confirm publication status, adding a note on the status (e.g., "v1 published", "no changes detected, draft deleted"):

- [x] eia176 - No changes, draft deleted.
- [x] eia191 - v10 published
- [x] eia757a - No changes, draft deleted
- [x] eia860 - No changes, draft deleted
- [x] eia860m - v24.0 published
- [x] eia861 - 2023 ER data released August 7th. v10.0 published
- [x] eia923 - v21.0 published
- [x] eia930 - v8.0 published
- [x] eiawater - No changes detected, draft deleted
- [x] eia_bulk_elec - v12.0 published
- [x] epacamd_eia - No changes detected, draft deleted
- [x] ferc1 - Updates are non-existent partition updates, no other changes observed. Draft deleted
- [x] ferc2 - Deleted archive by accident! Recreated, updates are non-existent partition updates, with 2023 data changes observed. v11.0 published
- [x] ferc6 - Updates are non-existent partition updates, with new data for 2023. v8.0 published.
- [x] ferc60 - Updates are non-existent partition updates, no other changes observed. Draft deleted
- [x] ferc714 - Updates are non-existent partition updates, no other changes observed. Draft deleted
- [x] mshamines - v9.0 published
- [x] nrelatb - No changes detected, draft deleted
- [x] phmsagas - v9.0 published
- [x] epacems - v14.0 published

Validation failures

For each run that failed because of validation test failures (seen in the GHA logs), add it to the tasklist. Download the run summary JSON by going into the "Upload run summaries" tab of the GHA run for each dataset, and follow the link. Investigate the validation failure.

If the validation failure is deemed ok after manual review (e.g., Q2 of 2024 data doubles the size of a file that only had Q1 data previously, but the new data looks as expected), go ahead and approve the archive and leave a note explaining your decision in the task list.

If the validation failure is blocking (e.g., file format incorrect, whole dataset changes size by 200%), make an issue to resolve it.

- [x] eia930 - Validation failure investigated, due to change in file size of new half year of data. v8.0 published

Other failures

For each run that failed because of another reason (e.g., underlying data changes, code failures), create an issue describing the failure and take necessary steps to resolve it.

- [x] eiaaeo - "Record has been deleted." Not sure why this issue has returned. Contacting Zenodo, but ran this manually against the latest record and no record has been updated. Addressed in #419
e-belfer commented 2 months ago

All archives have been reviewed and published!