search

gaul / s3proxy

Access other storage backends via the S3 API
Apache License 2.0
1.75k stars 228 forks source link

Cannot use S3Proxy with Azure Pod Identities #352

Open klolos opened 3 years ago

klolos commented 3 years ago

We are trying to deploy S3Proxy in a Kubernetes cluster on AKS in order to translate AWS S3-compatible requests performed by our own software to Azure Blob Storage requests. Even though passing an Azure storage account token to the JCLOUDS_CREDENTIAL variable works, we are having trouble authenticating using Azure Pod Identity credentials.

In a nutshell, Pod Identities allow pods running in an AKS cluster to retrieve a JWT from the Azure Instance Metadata Service and use it to assume a managed identity when accessing Azure services. We want to use this feature because it allows assigning fine-grained permissions to services running in the AKS cluster, and is considered best practice for applications running on AKS. See more information here:

https://docs.microsoft.com/en-us/azure/aks/use-azure-ad-pod-identity https://docs.microsoft.com/en-us/azure/aks/operator-best-practices-identity#use-pod-managed-identities

When trying to use this feature in combination with S3Proxy, we faced the following two problems:

  1. S3Proxy does not automatically retrieve and refresh Pod Identity credentials from the metadata service. This is a problem because these credentials are short-lived, and are expected to be refreshed regularly by the application.

  2. When passing a Pod Identity token retrieved from the metadata service to S3Proxy via the JCLOUDS_CREDENTIAL variable, requests towards S3Proxy fail with the following error:

    <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 400 </title>
</head>
<body>
<h2>HTTP ERROR: 400</h2>
<p>Problem accessing /. Reason:
<pre>    com.google.common.io.BaseEncoding$DecodingException: Unrecognized character: .</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

    What we think is happening is that jclouds is trying to base64 decode the Pod Identity token in order to sign the request. However, since this token is a JWT, decoding fails with the above error once it encounters the first dot.

Is there any way to overcome these problems and use S3Proxy with Pod Identities right now? Is there any plan to add support for them in the future?

gaul commented 3 years ago

jclouds will need to add support for Pod Identities and refreshing the credential. I recommend opening a JIRA issue although you will likely need to submit a pull request yourself.

Hackmeat commented 1 year ago

have u tried using the new Azure AD Workload Identity? Because the aad-pod-identity is deprecated since October 24th 2022 https://github.com/Azure/aad-pod-identity