App Framework: Integrate Azure SDK for Go for Azure Blob Client

[locmai]

Please select the type of request

Enhancement

Tell us more

Describe the request

• The current implementation of the pkg/splunk/client/azureblobclient.go is based on the Go net/http module, requiring manual construction of all requests instead of using the official Azure SDK for Go. We have observed a couple of issues related to the authentication requests, see Additional context section.
• Similar to what we have with AWS S3 client in pkg/splunk/client/awss3client.go

Expected behavior

• pkg/splunk/client/azureblobclient.go to use Azure SDK for Go

Splunk setup on K8S

• Simple Splunk on K8S in Azure wirh aadpodbinding and App Framework with Azure blob storage

Reproduction/Testing steps

• N/A

K8s environment

• Azure Kubernetes Service

Proposed changes(optional)

• Refactor the current AzureBlobClient to use armstorage.BlobContainersClient instead (or https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/storage/azblob)
• Refactor the updateAzureHTTPRequestHeaderWithIAM() and use azidentity.NewDefaultAzureCredential() for authentication and token refresh
• GetAppsList() could now implement and use https://github.com/Azure-Samples/azure-sdk-for-go-samples/blob/642ceb8c3db8b3d90f91f9db941ee462bc89624b/sdk/resourcemanager/storage/blob/main.go#L178C6-L178C23 instead
• See https://github.com/Azure-Samples/azure-sdk-for-go-samples/blob/main/sdk/resourcemanager/storage/blob/main.go

Additional context

We got connect: cannot assign requested address error message intermittently:

AzureBlob:GetAppsList   Failed to get http request authenticated    {"controller": "clustermanager", "controllerGroup": "enterprise.splunk.com", "controllerKind": "ClusterManager", "ClusterManager": {"name":"cm","namespace":"splunk"}, "namespace": "splunk", "name": "cm", "reconcileID": "c488e81f-1f94-4e8b-90f3-6cb32809c5dd", "Endpoint": "https://<redacted>.blob.core.windows.net/", "Bucket": "splunk-apps", "Prefix": "indexers/", "error": "Get \"http://169.254.169.254/metadata/identity/oauth2/token?api-version=2021-10-01&resource=https%3A%2F%2Fstorage.azure.com%2F\": dial tcp 169.254.169.254:80: connect: cannot assign requested address"}

And https://github.com/splunk/splunk-operator/issues/1356

[locmai]

I'd be happy to work on this and open a PR. Just want to confirm if this is on the right track.

[vivekr-splunk]

currently we are working on this change, hoping to release this with upcoming release. we will update you once we are done with the change.