gbif-norway / resolver-docker

A dockerised version of the GBIF.no resolver
Apache License 2.0
0 stars 2 forks source link

Provenance/history of resolvable objects #28

Open rukayaj opened 1 year ago

rukayaj commented 1 year ago

I fixed some bugs over the summer in the way we record the provenance/history/changes made to the data we publish. Endpoint: https://resolver.gbif.no/history/. I also added some additional search fields, so it is now possible to do more complex queries with dates.

It's possible to search for changes to particular data units/resolvable objects using the uuid like this https://resolver.gbif.no/history/?resolvable_object=e1dfb5b8-a4dc-4e51-9dc7-4c174a955411

If a resolvable object was not retrievable from the dwca, it's now stamped with a deleted_date, and such records can now be searched for like this https://resolver.gbif.no/?deleted_date_lte=2022-08-01

As discussed previously, we should add a new /cluster endpoint and do some kind of clustering logic where we suspect we have resolvable object ids which have "changed" (most likely suspects being the dwc triplets). So we could start by just looking for records where the entire dataset has a deleted_date stamp, and compare it with newly created datasets on the same run.