Support to Entra ID to get Storage Account Properties

[sergioaten]

Describe the bug When I try to fetch all the storage accounts using the command ./steampipe query "select * FROM azure_storage_account" --output line, I get an error message indicating that key-based authentication is not permitted on this storage account.

Steampipe version (steampipe -v) v0.23.2

Plugin version (steampipe plugin list) v0.59.0

To reproduce

1. Create a storage account in Azure that ONLY uses Entra ID for authentication.
2. Run the command ./steampipe query "select * FROM azure_storage_account" --output line

Expected behavior I expected to get a list of all storage accounts without any error.

Additional context The error message is: Error: azure: accounts.Client#GetServiceProperties: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="KeyBasedAuthenticationNotPermitted" Message="Key based authentication is not permitted on this storage account.\nRequestId:69914422-101e-0067-25b2-c563fa000000\nTime:2024-06-23T21:18:15.3109599Z" (SQLSTATE HV000). This issue seems to occur because I am using ONLY Entra ID authentication for this storage account.

[ParthaI]

Hello @sergioaten,

I looked into the issue and was able to replicate the problem on my end.

Here’s what I understand:

• According to the error message, we are encountering a forbidden error when querying this table.
• This error does not occur with list/get API calls but arises when attempting to retrieve the properties of the storage account using the GetServiceProperties API call.
• The error occurs because either the configuration for the storage account restricts access to property details or the caller lacks the necessary permissions to make this API call.
• Typically, we don’t handle forbidden errors within the plugin itself.
• If you want to avoid forbidden errors while querying the table, you can take the following steps:
  • Query only the columns that do not require extra hydrate calls to populate values, such as select name, id, type, access_tier, kind, sku_name, creation_time from azure_storage_account.
  • Configure the ignore_error_codes in your connection configuration file (azure.spc).

Example configuration:

connection "azure" {
  plugin = "azure"
  // Other configurations

  ignore_error_codes = ["KeyBasedAuthenticationNotPermitted"]
}

Thanks!

[sergioaten]

As far as I can see it happens in any Storage Account, whether access is controlled by Login ID or Access Key.

I have enough permissions in all the storage account that I am testing, since it is my environment, if I make the calls directly by API, I can make them without problem, is it necessary to add some option to the authentication? I'm authenticated by AZ CLI with az login

Something happens because in each request, the error changes and even in some, it shows me some Storage Account.

So... Why if I don't ignore this kind of errors it doesn't work?

[ParthaI]

Hello @sergioaten,

When you run the query select * from azure_storage_account, the following API calls are made:

1. To list all storage accounts within a subscription:

   • API: List API
   • Equivalent Azure CLI command: az storage account list

2. To get details of a specific storage account:

   • API: GetProperties API
   • Equivalent Azure CLI command: az storage account show --resource-group demo --name test732

3. For Storage Account Lifecycle Management Policy:

   • API: ManagementPoliciesClient.Get
   • Equivalent Azure CLI command: az storage account management-policy show --account-name mystorageaccount --resource-group myresourcegroup

4. To get Azure Storage Account Blob Properties:

   • API: BlobServicesClient.GetServiceProperties
   • Equivalent Azure CLI command: az storage account blob-service-properties show --account-name mystorageaccount --resource-group myresourcegroup

5. For Azure Storage Account Table Properties:

   • API: ServiceClient.GetProperties

6. For Azure Storage Account Blob Service Logging:

   • API: Client.GetServiceProperties

7. To get Azure Storage Account Queue Properties:

   • API: Client.GetServiceProperties

We are encountering a forbidden error for points 5, 6, and 7 when the Allow storage account key access is disabled, as the client is created using the storage account key. You can find more details here.

If we switch to using Entra ID for authentication, it may cause breaking changes due to different API packages. It's also unclear if we will need to authenticate using storage account keys in some cases.

I have pushed the changes by creating a client with Login ID or Access key to the branch issue-788 and mapped the column values according to the API response. I believe there should be no discrepancies in the results between the current plugin version and the plugin built from this branch. Please try it out and share your feedback.

Thanks!

[ParthaI]

Hey @sergioaten, did you have a chance to review the conversation above?

[ParthaI]

Hello @sergioaten, we are closing this issue Since we haven’t heard back from you in some time, we’re assuming that the issue has been resolved or that it’s no longer a concern.

As a result, we will proceed with closing this issue. However, if you still require assistance or have any further questions, please don't hesitate to reach out. We would be happy to reopen the issue and assist you further.

Thank you!