search

wellcomecollection / catalogue-pipeline

:oil_drum: The data pipeline services extracting & transforming data from our museum and collections.
https://developers.wellcomecollection.org/catalogue
MIT License
13 stars 2 forks source link

Handle removal of content from EBSCO #2560

Closed paul-butcher closed 4 months ago

paul-butcher commented 8 months ago

How do EBSCO notify us that we no longer have access to a resource? How do we get that through the pipeline.

Originally posted by @paul-butcher in https://github.com/wellcomecollection/catalogue-pipeline/issues/2556#issuecomment-1952533440

If deletion is only apparent by the absence of a record in an update, then we need a way to recognise that it is missing, so that we can mark our record as deleted.

paul-butcher commented 8 months ago

I currently have no idea how we are notified of deletion or removal of access (e.g. if we change what we subscribe to)

This ticket is here so that we don't forget to work it out.

paul-butcher commented 8 months ago

This may be an adapter job - compare current with previous, or with the content of a directory containing records cracked from the collection, and write a DELETED record.

paul-butcher commented 7 months ago

Part of https://github.com/wellcomecollection/platform/issues/5738

kenoir commented 4 months ago

Closing as done.