loris-imageserver / loris

Loris IIIF Image Server
Other
208 stars 87 forks source link

Add support for versioned content in VERSIONED S3 bucket #553

Open yarikoptic opened 1 year ago

yarikoptic commented 1 year ago

I believe interfacing data from S3 was added to LORIS recently. Some buckets could use versioning support to provide efficient way to host multiple version of the datasets. E.g it is done so by OpenNeuro.

If LORIS could reference specific version ID for specific file -- that would open better integration of LORIS with S3 for such projects. E.g. have a look at S3 bucket for openneuro which has multiple versions for different files, e.g.

❯ datalad ls -aL s3://openneuro.org/ds004288/README
Connecting to bucket: openneuro.org
[INFO   ] S3 session: Connecting to the bucket openneuro.org anonymously 
Bucket info:
  Versioning: S3ResponseError: 403 Forbidden
     Website: S3ResponseError: 403 Forbidden
         ACL: <Policy: openneurocommon (owner) = FULL_CONTROL, http://acs.amazonaws.com/groups/global/AllUsers = READ, rblair2 = FULL_CONTROL, http://acs.amazonaws.com/groups/global/AllUsers = READ_ACP>
ds004288/README 2022-12-08T16:32:24.000Z 1085 ver:2j0mc8EgCLUExJqdiXICySNR_EU.5dqG  acl:AccessDenied  http://openneuro.org.s3.amazonaws.com/ds004288/README?versionId=2j0mc8EgCLUExJqdiXICySNR_EU.5dqG [OK]
ds004288/README 2022-12-08T16:32:24.000Z DeleteMarker
ds004288/README 2022-12-08T16:32:21.000Z  835 ver:_7xU3APkKeX2XWZK_7CtAj6icCSQSa0a  acl:AccessDenied  http://openneuro.org.s3.amazonaws.com/ds004288/README?versionId=_7xU3APkKeX2XWZK_7CtAj6icCSQSa0a [OK]
ds004288/README 2022-12-08T16:32:20.000Z DeleteMarker
ds004288/README 2022-12-08T16:32:19.000Z  512 ver:vmWiK82MmFwbSvusTC4QAD523Ex8TOma  acl:AccessDenied  http://openneuro.org.s3.amazonaws.com/ds004288/README?versionId=vmWiK82MmFwbSvusTC4QAD523Ex8TOma [OK]
ds004288/README 2022-12-08T16:32:18.000Z DeleteMarker
...

attn @breen0074