search

wellcomecollection / platform

Wellcome Collection Digital Platform
https://developers.wellcomecollection.org/
MIT License
48 stars 10 forks source link

Transformer should report cataloguing errors to a separate queue #2562

Closed alexwlchan closed 3 years ago

alexwlchan commented 6 years ago

Some errors in the transformer are actually cataloguing errors – for example, having both 260 and 264 fields on a record (#2280).

In this case, as well as sending them to the DLQ, the transformer should send a message to a separate queue that says “this record is catalogued badly, here is why, please do a manual fix”, and that can be passed to somebody with Sierra access.

alexwlchan commented 3 years ago

We've been adding warning logs and making a best-effort attempt to transform rather than dropping the works. This would be nice, but I don't think it's on the horizon. Closing.