search

duckdb / duckdb_aws

MIT License
34 stars 12 forks source link

s3_region not set through the extension to access a S3 file #6

Closed stephaniewang526 closed 10 months ago

stephaniewang526 commented 10 months ago

Given that the httpfs extension still uses v4 signing for auth so AWS_REGION is a required paramter to be passed to access a S3 file, we should set it through the extension so that users don't have to separately set it using the SET command.

My experimentation with this code snippet worked for extracting AWS_REGION from my local ~/.aws/credentials file but I'm unfamiliar with AWS SDK so just a suggestion:

#include <aws/core/client/ClientConfiguration.h>

...
    string ret;
    if (!credentials.IsExpiredOrEmpty()) {
        config.SetOption("s3_access_key_id", Value(credentials.GetAWSAccessKeyId()));
        config.SetOption("s3_secret_access_key", Value(credentials.GetAWSSecretKey()));
        // Extract and set AWS_REGION
        Aws::Client::ClientConfiguration s3_config(profile.c_str());
        auto region = s3_config.region;
        config.SetOption("s3_region", Value(region));
        config.SetOption("s3_session_token", Value(credentials.GetSessionToken()));
        ret = credentials.GetAWSAccessKeyId();
    }
...
samansmink commented 10 months ago

fixed by https://github.com/duckdblabs/duckdb_aws/pull/8. Thanks for the help!