[Feature] Dataset Versioning

[jcace]

• Support for datasets to grow over time, with an incrementing version number/hash that changes whenever content is added
• This should probably come after integration with Ptolemy so the versioning is handled end-to-end
• Build in some additional flexibility to allow users to replicate a specific version first, and see which carfile belongs to which version

[jcace]

as per discussion with @schreck , this should be able to be handled completely by Ptolemy, DDM does not need to be aware of the versioning.

[anjor]

Interesting. I had built this on top of estuary using the collections api:

https://github.com/application-research/pestuary/blob/main/src/pestuary/versioned_uploads/versioned_uploads.py