search

duckdb / duckdb_aws

MIT License
34 stars 12 forks source link

Using duckdb aws within kubernetes with IAM role and service account #27

Closed javiermas closed 4 months ago

javiermas commented 4 months ago

I'm trying to use duckdb to query some data in s3. Locally, I use the function load_aws_credentials(), which injects the aws credentials I have in my config file. In our kubernetes cluster, we authenticate with aws via service account and IAM roles. I've checked the pod has access to s3 through the CLI, but I get 403 errors when trying to query some data on s3 through duckdb, either with or without load_aws_credentials. Any idea how to proceed?

AlejandroUPC commented 4 months ago

I am also very interested in this and expected to be already on the 0.9.X versions on Duckdb, seems to be some delay?

javiermas commented 4 months ago

Still not able to do this, even though I'm using the create secret process deployed in version 0.10.0 of duckdb. The create secret process doesn't work even locally because of a different error: IO Error: Connection error for HTTP HEAD to 'XXX'