Open RubenVerborgh opened 9 years ago
Thanks for making this feature suggestion! Laurens and I have pondered taking dataset dynamics into account for quite a while now, but we haven't been able to pick the topic up yet. There is a project out there that does track dataset dynamics (pardon the name): http://swse.deri.org/dyldo/
The way I see it, we need the following technical innovations in order to have the LOD Laundromat take dataset dynamics into account:
All in all, I believe the above changes are realistic for a potential future version of the LOD Laundromat (see milestone).
PS: A more difficult, related topic is that of archiving the LOD Cloud. Saving multiple versions of the same data document over time requires (1) a technique for storing only the diff between consecutive files, and (2) a generous amount of disk space we are currently lacking.
At the moment, it seems the LOD Laundromat settles for one version of a dataset. However, dataset change over time. Should the Laundromat also update its version? This requires a mechanism to find out whether datasets are different. (Etag? File size? Last-Modified?)
Furthermore, it should be decided what is done then:
Most importantly, the Laundromat can give dataset owners an incentive to clean their data. However, if the old version remains featured prominently, this can give the wrong impression. E.g., people can spot an outdated version of a dataset on the Laundromat, and mistakenly assume the quality issues are still there, even though the original has been updated.