delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python
https://delta-io.github.io/delta-rs/
Apache License 2.0
1.98k stars 365 forks source link

[deltalake_aws] Allow passing AWS_SESSION_TOKEN and other credentials more directly to DynamoDB client. #2497

Open adamfaulkner-at opened 1 month ago

adamfaulkner-at commented 1 month ago

Description

My understanding of the code here is that AWS_SESSION_TOKEN, AWS_SECRET_ACCESS_KEY, and AWS_ACCESS_KEY_ID are assigned to environment variables, and later inferred by the S3 client and the DynamoDB client. It would be nicer if they could be passed directly to these clients without being part of the process's environment.

I think an ideal solution would allow me to configure my own object store and dynamo client, and provide those when registering a table.

Use Case

My process is a big monolith with a bunch of stuff in it, completely unrelated to me. Many different identities are used in this process for many different purposes. Environment variables are a single global namespace, so it would be dangerous to plug my credentials in it.

Related Issue(s)