For data providers with sufficient metadata on authors and dates of changes (e.g. probably figshare is such, ref: #20), ambitious we (or someone else) could make a wild attempt to make such crawled datasets fully reproducible!
If our dataset uuid would be pre-seeded (or deterministically produced out of the url or some other persistent information relevant for that dataset), we then could force authors and commit dates to correspond to information from the providers, so whenever anyone else would 'crawl' it again - he would get bit to bit (at git/annex level) identical dataset.
That could then provide a sample/opportunity to incorporate some of those providers directly into datalad install, so ppl could just do datalad install https://figshare.com/articles/BOLD5000/6459449 and always get the same/interoperable dataset (git/git-annex repo)
For data providers with sufficient metadata on authors and dates of changes (e.g. probably figshare is such, ref: #20), ambitious we (or someone else) could make a wild attempt to make such crawled datasets fully reproducible!
If our dataset uuid would be pre-seeded (or deterministically produced out of the url or some other persistent information relevant for that dataset), we then could force authors and commit dates to correspond to information from the providers, so whenever anyone else would 'crawl' it again - he would get bit to bit (at git/annex level) identical dataset.
That could then provide a sample/opportunity to incorporate some of those providers directly into
datalad install, so ppl could just dodatalad install https://figshare.com/articles/BOLD5000/6459449and always get the same/interoperable dataset (git/git-annex repo)