apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.2k stars 3.19k forks source link

[Missing feature] S3 Credentials: Cannot use system IAM credentials instead of hardcoded ones #35928

Open tgho-brrrr opened 2 months ago

tgho-brrrr commented 2 months ago

Search before asking

Description

In the docs, it is mentioned that we can create an S3 catalog to enhance security by avoiding hardcoded credentials, and use system credentials.

However, it seems like I cannot create a catalog with type="S3" (ERROR 1105 (HY000): errCode = 2, detailMessage = Unknown catalog type: s3).

It is also not possible to create a resource without omitting the access key and the secret key: ERROR 1105 (HY000): Unexpected exception: Access key ID cannot be blank.

Solution

IMHO, no need for an S3 catalog here, contrary to what the docs suggest.

The simplest solution would be to allow to create a resource without access key and secret key. The logic to automatically search for the credential chain is already handled it the java AWS SDK, and also seem to be handled on Doris side:

In this file: S3Util.java

Are you willing to submit PR?

Code of Conduct

tgho-brrrr commented 2 months ago

Seems to be handled in the backend as well, here.

But might need to use the default constructor here, and get rid of this.