search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.45k stars 3.01k forks source link

Unable to load AWS credentials when querying from Iceberg Nessie Catalog #19504

Closed domingozhang closed 1 year ago

domingozhang commented 1 year ago

Release Trino 429 Nessie Catalog + Iceberge + DELL ECS storage

Connector configuration: connector.name=iceberg iceberg.catalog.type=nessie iceberg.nessie-catalog.uri=http://ecshost:19120/api/v1 iceberg.nessie-catalog.default-warehouse-dir=/tmp

In client, I can run the following command:

show catalogs; use iceberge.schema; create table test01(a int,b int); show tables;

However, when I ran a select statement, notified an error:

trino:dwd> select * from test01; Query 20231024_012322_00005_sw687, QUEUED, 0 nodes, 0 splits ... Query 20231024_012322_00005_sw687 failed: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@5ef0a49: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@6904b70d: Failed to connect to service endpoint: ]

I cannot get any guide about how to config AWS Credentials in user manual for Iceberg connector with Nessie catalog. Did I miss anything? Best regards.

ebyhr commented 1 year ago

Please take a look at S3 config properties https://trino.io/docs/current/connector/hive-s3.html

domingozhang commented 1 year ago

thanks @ebyhr I use iceberg connector. Shall I config another hive connector to connect S3?

ebyhr commented 1 year ago

No, those properties are available in Iceberg connector as well.

domingozhang commented 1 year ago

hi @ebyhr Still have some issues when querying a table. My S3 configurations are as following: hive.s3.aws-access-key= $my_access_key hive.s3.aws-secret-key= $my_secret_key hive.s3.endpoint=http://$my_end_point_ip:port hive.s3.path-style-access=true hive.s3.ssl.enabled=false

When querying, I got this error message trino:dwd> select * from test

Query 20231026_020624_00005_tr6ns, FAILED, 1 node Splits: 1 total, 0 done (0.00%) 4:59 [0 rows, 0B] [0 rows/s, 0B/s] Query 20231026_020624_00005_tr6ns failed: Error opening Iceberg split s3://nessie/dwd/dwd_t01_bnk/1ac9e7b8-aadf-50a1-83ce-efb541728900/3_0_0.parquet (offset=0, length=4210): Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@3799f185: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@4825e22e: Failed to connect to service endpoint: ]

After tried configure environment variables, it was still failed. Could you please take a look at my issue? thanks