search

stevearc / pypicloud-docker

Docker image for pypicloud
MIT License
86 stars 34 forks source link

How to use private s3 bucket in pypicloud ? #34

Open nikhilagrawal577 opened 3 years ago

nikhilagrawal577 commented 3 years ago

Hello,

I wanted to use s3 bucket for storing my python libraries. I cross checked the document https://pypicloud.readthedocs.io/en/latest/topics/storage.html#storage-bucket. I configured the details.

As we have a private s3 bucket and URL is not the same. [http://s3.amazonaws.com/] However how do we pass our host which takes s3 data ?

stevearc commented 3 years ago

What is going wrong? Pypicloud uses boto to generate signed URLs to the packages. You shouldn't have to manually deal with any URLs yourself. Have you configured the IAM policy for the user?

ddelange commented 2 years ago

signed urls are generated with hardcoded amazonaws.com host.

there's a PR open, one of which side effects is

does not need to generate signed urls to perform a streamed response (rather re-uses the storage client) when stream_files is true

instead, it will re-use the boto client generated from the config you pass. so enabling stream_files will allow you to bypass the hardcoded host in the signed urls.

hope that helps!