search

MichaelMt66 / open-source-lakehouse

3 stars 2 forks source link

The AWS Access Key Id you provided does not exist in our records #2

Closed alberttwong closed 5 months ago

alberttwong commented 5 months ago

error on docker compose startup

hive-metastore       | Exception in thread "main" MetaException(message:Got exception: java.nio.file.AccessDeniedException hive-warehouse-data: doesBucketExistV2 on hive-warehouse-data: com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: C1HKGMM7Q4BTWY6C; S3 Extended Request ID: oFqB9pRR+uGHHbqO/o6n1x1ciTkirzUVB/pLJB/DUT//V+8EdsE1U+hkHs6LstMGbT5pZYqxpKXx6j45k9quKJToB5sXXr2h), S3 Extended Request ID: oFqB9pRR+uGHHbqO/o6n1x1ciTkirzUVB/pLJB/DUT//V+8EdsE1U+hkHs6LstMGbT5pZYqxpKXx6j45k9quKJToB5sXXr2h:InvalidAccessKeyId)
hive-metastore       |  at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:84)
hive-metastore       |  at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
hive-metastore       |  at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8661)
hive-metastore       |  at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8656)
hive-metastore       |  at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8926)
hive-metastore       |  at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8843)
hive-metastore       |  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore       |  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
hive-metastore       |  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore       |  at java.base/java.lang.reflect.Method.invoke(Method.java:567)
hive-metastore       |  at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
hive-metastore       |  at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
hive-metastore       | Caused by: MetaException(message:Got exception: java.nio.file.AccessDeniedException hive-warehouse-data: doesBucketExistV2 on hive-warehouse-data: com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: C1HKGMM7Q4BTWY6C; S3 Extended Request ID: oFqB9pRR+uGHHbqO/o6n1x1ciTkirzUVB/pLJB/DUT//V+8EdsE1U+hkHs6LstMGbT5pZYqxpKXx6j45k9quKJToB5sXXr2h), S3 Extended Request ID: oFqB9pRR+uGHHbqO/o6n1x1ciTkirzUVB/pLJB/DUT//V+8EdsE1U+hkHs6LstMGbT5pZYqxpKXx6j45k9quKJToB5sXXr2h:InvalidAccessKeyId)
hive-metastore       |  at org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:168)
hive-metastore       |  at org.apache.hadoop.hive.metastore.Warehouse.getFs(Warehouse.java:117)
hive-metastore       |  at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:141)
hive-metastore       |  at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:147)
hive-metastore       |  at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:160)
hive-metastore       |  at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultCatalog(HiveMetaStore.java:731)
hive-metastore       |  at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:768)
hive-metastore       |  at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:538)
hive-metastore       |  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore       |  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
hive-metastore       |  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore       |  at java.base/java.lang.reflect.Method.invoke(Method.java:567)
hive-metastore       |  at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
hive-metastore       |  at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
hive-metastore       |  at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80)
hive-metastore       |  ... 11 more
hive-metastore       | 2024-02-01 18:21:26,416 shutdown-hook-0 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
hive-metastore       | 2024-02-01 18:21:26,421 shutdown-hook-0 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
hive-metastore       | 2024-02-01 18:21:26,446 shutdown-hook-0 WARN Unable to register Log4j shutdown hook because JVM is shutting down. Using SimpleLogger
Gracefully stopping... (press Ctrl+C again to force)
dependency failed to start: container hive-metastore exited (1)
alberttwong commented 5 months ago

It looks like the AWS key is been stored in your images. I can't seem to start it up enough to remove them.

alberttwong commented 5 months ago

ahh.. it's in .env within the infra folder.