cc-archive / cccatalog

[PROJECT TRANSFERRED] Mapping the commons towards an open ledger and cc search.
https://github.com/WordPress/openverse-catalog
MIT License
63 stars 60 forks source link

[Infrastructure] Create a workflow for alerting about newly added categories (unit codes) in Smithsonian #451

Closed ChariniNana closed 4 years ago

ChariniNana commented 4 years ago

Current Situation

The images from the Smithsonian API are categorised based on a unit code value, and the unique unit code values can be retrieved from calling the following endpoint: https://api.si.edu/openaccess/api/v1.0/terms/unit_code?api_key=REDACTED

We intend to use currently known unit code values for retrieving the sub providers within Smithsonian, and if a previously unknown unit code is added from their end, we would be unaware of the change and thus ignore it in our relevant implementation (or attempting to update sub providers would throw an error). Therefore, it is necessary to have a mechanism of checking whether a unit code update has occurred in the Smithsonian API.

Suggested Improvement

Add a seperate workflow which is triggered once or few times a day to retrieve the unit codes from the Smithsonian API and raise an alert if previously unseen unit codes are encountered.

Benefit

Having a seperate workflow for retrieving the unit codes from the Smithsonian API would help minimise cluttering the sub provider retrieval logic and provides a cleaner way of alerting about changes in Smithsonian unit codes which should be updated from our end.