search

NVIDIA / aistore

AIStore: scalable storage for AI applications
https://aistore.nvidia.com
MIT License
1.23k stars 164 forks source link

Question Regarding S3 Compatible Backend Provider #124

Closed jgould22 closed 1 year ago

jgould22 commented 1 year ago

I am working through https://github.com/NVIDIA/aistore/blob/master/deploy/prod/docker/single/README.md to get a basic test setup going.

I would like to use an S3 Compatible store(MinIO or Ceph for example) however from what I can tell the aws provider does not have a setting (presumably because the underlying library does not provide this, see https://github.com/aws/aws-sdk/pull/230)

Is there a way I can set the back-end provider to an s3 compatible store?

alex-aizman commented 1 year ago

See:

Note also that any backend-specific credentials must be available at cluster startup, which is not easy to achieve when you have a pre-built container. Might make sense to, first, experiment with running aistore from the source where you don't have those restrictions.

jgould22 commented 1 year ago

Thank you for the quick response.