search

seung-lab / cloud-files

Threaded Python and CLI client library for AWS S3, Google Cloud Storage (GCS), in-memory, and the local filesystem.
BSD 3-Clause "New" or "Revised" License
36 stars 8 forks source link

Support for AWS_SESSION_TOKEN #101

Closed madiganz closed 4 months ago

madiganz commented 4 months ago

I recently ran into an issue where I could check if a file existed using boto3, but it would fail for CloudFiles. This is because I was required to use AWS_SESSION_TOKEN and the current secrets retrieval and boto3 initialization does not support this. I modified some files locally to support this and it seems to work well with minimal code changes. I am happy to create a PR for this. Let me know your thoughts

william-silversmith commented 4 months ago

This sounds useful, yes please make a PR ☺️

On Mon, Mar 4, 2024, 6:55 PM madiganz @.***> wrote:

I recently ran into an issue where I could check if a file existed using boto3, but it would fail for CloudFiles. This is because I was required to use AWS_SESSION_TOKEN and the current secrets retrieval and boto3 initialization does not support this. I modified some files locally to support this and it seems to work well with minimal code changes. I am happy to create a PR for this. Let me know your thoughts

— Reply to this email directly, view it on GitHub https://github.com/seung-lab/cloud-files/issues/101, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATGQSNM56WUXZEI6EIBXTLYWUCV3AVCNFSM6AAAAABEGEDZSKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE3DQMBRGA2DEMA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

madiganz commented 4 months ago

@william-silversmith created the PR. I did not add any tests for this, but I am happy to add a test that validates that aws_credentials contain the AWS_SESSION_TOKEN if you would like.

william-silversmith commented 4 months ago

Yea, adding that test would be helpful because I'll probably forget about this. That way if I screw something up it'll alert me.

Thank you!

Will

On Mon, Mar 4, 2024, 8:39 PM madiganz @.***> wrote:

@william-silversmith https://github.com/william-silversmith created the PR. I did not add any tests for this, but I am happy to add a test that validates that aws_credentials contain the AWS_SESSION_TOKEN if you would like.

— Reply to this email directly, view it on GitHub https://github.com/seung-lab/cloud-files/issues/101#issuecomment-1977786477, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATGQSPM6PBBYNF22NNXKGLYWUO5HAVCNFSM6AAAAABEGEDZSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXG44DMNBXG4 . You are receiving this because you were mentioned.Message ID: @.***>

madiganz commented 4 months ago

I added a test for the verification

madiganz commented 4 months ago

Please let me know when you create a release with this change.

william-silversmith commented 4 months ago

New release uploaded!