search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.42k stars 1.27k forks source link

Add the support for DefaultAzureCredential for ADLSGen2PinotFS #9805

Open snleee opened 1 year ago

snleee commented 1 year ago

Currently, S3PinotFS implementation falls back to initialize credential using DefaultCredentialsProvider when the access key and secret is not provided by the pinot config. In this way, we allow users to set env variables like AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

For ADLS Gen2 Pinot FS, we expect users to pass accessKey and accountName via pinot config. We should use default credential provider when those information is not given.

Some references to look:

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-java?tabs=powershell%2Cmanaged-identity%2Croles-azure-portal%2Csign-in-azure-cli https://blog.jongallant.com/2021/08/azure-identity-101/

snleee commented 1 year ago

@xiangfu0

ceekay47 commented 5 months ago

Hi @snleee, is this still a valid issue? If so, can I pick this up?

Jackie-Jiang commented 5 months ago

I think this is supported with #9855. @snleee Can you please verify?