trinodb / charts

Apache License 2.0
151 stars 173 forks source link

Unable to run queries with deployed Trino on EKS and AWS Glue #200

Closed acesir closed 3 months ago

acesir commented 3 months ago

We have configured Glue database with a sample table all pointing to S3. Using DBeaver to connect and we see the hive metastore as well as the newly created table. When we attempt to query the table and a job spawns it fails automatically with:

` Error Type EXTERNAL
Error Code HIVE_UNKNOWN_ERROR (16777221)
e io.trino.spi.TrinoException: The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 1AHXWGQGSP5ZTCZC; S3 Extended Request ID: 47yhatv5V1taSDBv+rUPyeKTebF8i6phv8OHrtbqdjHNuSO8NRfMjwuTTfxbeTxG1Vx2m+s42tA=; Proxy: null)

`

Inside of helm chart we have this set but it seems running any queries with the current helm chart fails.

additionalCatalogs:
  hive: |
    connector.name=hive
    hive.metastore=glue
    hive.metastore.glue.region=us-gov-west-1
    hive.metastore.glue.default-warehouse-dir=s3://MY-BUCKET/data/
    hive.metastore.glue.aws-access-key=KEY
    hive.metastore.glue.aws-secret-key=SECRET
    hive.s3.aws-access-key=KEY
    hive.s3.aws-secret-key=SECRET

Did we miss adding something or is this a bug we are hitting? Thanks

nineinchnick commented 3 months ago

The error message says your credentials (access key and secret key) are invalid. Please verify them.

acesir commented 3 months ago

The error message says your credentials (access key and secret key) are invalid. Please verify them.

Why was this closed right away? Our credentials are correct otherwise we would not even be able to connect to AWS Glue. And we verified on multiple occasions the credentials have correct permissions for S3 and everything else required.

nineinchnick commented 3 months ago

I don't see anything here related to the Trino Helm Chart. Try asking for help on the Trino Slack.

You're setting a specific region for Glue, maybe you also need to set it for Hive, as hive.s3.region: https://trino.io/docs/current/object-storage/legacy-s3.html#s3-configuration-properties