uoregon-libraries / newspaper-curation-app

Suite of front- and back-end tools for the curation of digitized newspaper materials
Apache License 2.0
8 stars 1 forks source link

Check for batch existence automatically #337

Closed jechols closed 1 month ago

jechols commented 2 months ago

This supports #336, making this a sub-sub-issue of #330.

Instead of having users flag a batch as live or purged or whatever, we should just make those checks part of various jobs.

Unlike typical pipelines, however, we're going to want to make sure users can still see what's going on. They need to know that a purge or load is necessary in the first place, since that isn't automated.

As I write this I think maybe we want automated sshing or something first. I don't want to drop what I've started here, but if automated sshing isn't awful, it makes more sense to do that first.

jechols commented 1 month ago

I think the upcoming ONI Agent integration will make this unnecessary. Ideally we'd actually verify batch loads and purges, but perfection is costly and it's time to move on. Sending off a command and trusting its successful completion to mean all is well... that's definitely in the "good enough" realm.