search

delta-io / kafka-delta-ingest

A highly efficient daemon for streaming data from Kafka into Delta Lake
Apache License 2.0
337 stars 72 forks source link

Question: How to use an EC2 Instance role for accessing S3/DynamoDB? #111

Closed xenji closed 1 year ago

xenji commented 2 years ago

I cannot find any reference to the default credential chain in the code, can anyone help me?

mosyp commented 2 years ago

Hi! It's managed by delta-rs and rusoto-s3, the default instance creds should work just fine. In addition you can even assume role via sts, please see https://github.com/delta-io/delta-rs/blob/main/rust/src/storage/s3/mod.rs#L30-L86 and https://github.com/delta-io/delta-rs/blob/main/rust/src/storage/s3/mod.rs#L398