catalyst-cooperative / pudl-archiver

A tool for capuring snapshots of public data sources and archiving them on Zenodo for programmatic use.
MIT License
5 stars 2 forks source link

Automate Zenodo archiving #61

Open jdangerx opened 1 year ago

jdangerx commented 1 year ago

This issue tracks all the tasks required to get our automated archiving system up and running.

Scope

Out of Scope

- [ ] #77 
- [ ] #79 
- [x] Add production credentials to workflow
- [ ] #73 
- [ ] #75 
- [ ] #76 
- [ ] #74 
- [ ] #60
- [ ] #70 
- [ ] https://github.com/catalyst-cooperative/pudl-archiver/issues/80
- [ ] https://github.com/catalyst-cooperative/pudl-archiver/issues/81
- [ ] https://github.com/catalyst-cooperative/pudl-archiver/issues/144
- [ ] https://github.com/catalyst-cooperative/pudl-archiver/issues/276
e-belfer commented 8 months ago

@zaneselvans Do we feel satisfied closing this? We've changed the scope to move away from per-archive conditions since we originally wrote this, but I've closed #276

zaneselvans commented 8 months ago

It seems like the base level "automate" is done, and we could probably gather together various other issues that improve the validation of archives and any archiving bugs we run into.

Can we close the bugs that are listed here now, since they don't seem to have been causing issues? I guess that's just #74 & #81. We should figure out how we want to deal with drafts and continue to make the success/failure conditions more robust.