Add last data change date to dataset API

[gbif-portal]

GBIF [Backbone] dataset page shows confusing information

The GBIF backbone dataset should omit these keys:

• Failed attempts since last successful ingestion
• Last failed attempt
• Last successful ingestion
• Last ingestion with changes
• Ingestion completion reasons
• Ingestions in total

since it isn't ever ingested, except when we had a bug. (The "trusted contact" block is fine.)

More generally, "Last Modified" is showing the last time the registry metadata (description etc) was changed, which isn't usually the most interesting thing.

For all datasets, that should probably be latest(dataset.last_modified, dataset.last_ingestion_with_changes).

I will change the last modification data for this manually, and @mdoering might remember this as part of the backbone release.

---

User provided contact info: @MattBlissett System: Firefox 59.0.0 / Ubuntu 0.0.0 User: See in registry Referer: https://www.gbif.org/dataset/d7dddbf4-2cf0-4f39-9b2a-bb099caae36c Window size: width 1200 - height 1792 API log&_a=(columns:!(_source),index:'prod-varnish-',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'response:%3E499')),sort:!('@timestamp',desc))) Site log&_a=(columns:!(_source),index:'prod-portal-',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'response:%3E499')),sort:!('@timestamp',desc))) System health at time of feedback: OPERATIONAL

[MortenHofft]

I see what you mean.

Should https://api.gbif.org/v1/dataset/d7dddbf4-2cf0-4f39-9b2a-bb099caae36c/process then also be changed since it is never ingested/crawled?

"startedCrawling": "2016-05-12T22:00:00.000+0000"

[MortenHofft]

thanks @MattBlissett

More generally, "Last Modified" is showing the last time the registry metadata (description etc) was changed, which isn't usually the most interesting thing.

No it isn't. It is however what the API returns and he general principle has been that the website should reflect the API. The APIs don't expose when data was last changed. Maybe we should add that?

We recently added an attempt to extract last data change from the process endpoint (but as Tim says that isn't really intended for consumption). And now show it in the registry bit. I agree it is important.

Perhaps we could include it in the dataset/key endpoint? It seems important for any user of the dataset API

[MortenHofft]

I've added last data change to the summary and removed backbone registry information from the UI.

I'm relabelling as API so that the last data change can be added to the API