DPLA bulk data comparison dependent on ingestion version

MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform

MIT License

26 stars 11 forks source link

DPLA bulk data comparison dependent on ingestion version #381

Open ghukill opened 5 years ago

ghukill commented 5 years ago

As Michigan moved from Ingestion1 --> Ingestion3, looks as though format of bulk data written to S3 has changed (unsurprisingly).

Not acting on for now, but worth noting.

antmoth commented 5 years ago

It looks like this should be something we can address, at the point at which it becomes important enough to sink time into.

I assume this is relevant: https://digitalpubliclibraryofamerica.atlassian.net/wiki/spaces/TECH/pages/5931056/Database+export+files

antmoth commented 5 years ago

Because this relies on the record_id being the same, this may need to be rethought. Also, how valuable it is depends on how valuable other hubs would consider it.