search

dandi / dandi-archive

DANDI API server and Web app
https://dandiarchive.org
14 stars 10 forks source link

Use Versioned MinIO buckets #970

Closed jjnesbitt closed 2 years ago

jjnesbitt commented 2 years ago

Our S3 buckets are versioned, but our local MinIO buckets are unversioned, leading to discrepancies between the two. We should use versioned MinIO buckets to address this. Relevant MinIO docs: https://docs.min.io/docs/minio-bucket-versioning-guide.html

jjnesbitt commented 2 years ago

At the moment this seems infeasible for a couple reasons.

The first being that django-minio-storage has the minio python client version pinned to < 7, and version 7.0.0 contains the versioning change.

The second being that versioning isn't supported on single-disk setups (what we use for development/testing). It's only supported in distributed setups.

So for now I suppose we'll just have to do our best with any differences between versioned and non-versioned buckets.