credentials set through env vars are not loaded

The extension currently seems to only load and set credentials saved in the local ~/.aws/credentials file. However, the AWSCredentialsProviderChain used (ProfileConfigFileAWSCredentialsProvider and DefaultAWSCredentialsProviderChain) should also have the ability to extract AWS credentials from the environment variables.

Steps to reproduce:

export AWS_ACCESS_KEY_ID="AKIA3..." 
export AWS_SECRET_ACCESS_KEY="XoH..."
export AWS_REGION=“us-east-1”

Check ~/.aws/credentials is empty. Then:

stephaniewang@Stephanies-MacBook-Pro duckdb_aws % ./build/debug/duckdb
v0.8.2-dev2700 acbbfe0e79
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D load aws;
D load httpfs;
D select * from read_parquet('s3://motherduck-dev-test-bucket/pypi.small.parquet');
Error: HTTP Error: HTTP GET error on 'https://motherduck-dev-test-bucket.s3.amazonaws.com/pypi.small.parquet' (HTTP 403)
D CALL load_aws_credentials();
┌────────────┐
│ loaded_key │
│  varchar   │
├────────────┤
│            │
└────────────┘
D select * from read_parquet('s3://motherduck-dev-test-bucket/pypi.small.parquet');
Error: HTTP Error: HTTP GET error on 'https://motherduck-dev-test-bucket.s3.amazonaws.com/pypi.small.parquet' (HTTP 403)

(I'm using a locally debug build DuckDB which is the version in the submodule and locally debug build AWS extension)

Ideally, we should be able to also extract AWS credentials from the environment variables and these should take precedence over what is being saved in ~/.aws/credentials.

Thanks for reading!

duckdb / duckdb_aws

credentials set through env vars are not loaded #7