Alias Request: Microsoft.Storage/storageAccounts.publicAccess

[klawrie]

Scenario: We need to audit the Storage Account if blob or container public access is enabled.

CLI: az storage container list --account-name [containername]--query "[*].properties.publicAccess" GUI: StorageAccounts/Account/Containers/AccessPolicy - 'Public access level.

This returns "blob" or "container" if access is enabled, or null if public access isn't enabled.

This is required for Azure CIS Benchmark 3.7

Definition:

{ "if": { "field": "type", "equals": "Microsoft.Storage" }, "then": { "effect": "auditIfNotExists", "details": { "type": "Microsoft.Storage/storageAccounts", "name": "default", "existenceCondition": { "field": "Microsoft.Storage/storageAccounts/container.blobPublicAccess", "equals": "" } } } }

[mcollis2]

We also are looking for this as it will prevent our enterprise from accidentally enabling access to data to the public unknowingly! Is there any update on this issue?

[mcollis2]

Here is the json reference to the property too:

{
  "type": "Microsoft.Storage/storageAccounts/blobServices/containers",
  "apiVersion": "2018-02-01",
  "name": "[concat(variables('accountName'), '/default/', variables('containerName')]",
  "dependsOn": [
    "[variables('accountName')]"
  ],
  "properties":{
      "publicAccess":"None"
  }
  }

[Yliche]

This is extremely important. This would be a data breach if somebody misconfigures a blob container to be publicly accessible. With DevOps teams owning their own infrastructure, you can only educate them so much. We can't put the guardrails up to completely prevent ANY public blob container from ever occurring.

With all of the misconfigured storage containers in AWS resulting in data breaches, I can't fathom how this isn't the very top priority to prevent anybody from accidentally making a container public with sensitive data.

[retux]

IMHO it would be a great feature to add from a security standpoint. Looking forward for that. In the meantime, is there any workaround at least to audit blob containers with public permissions?

[mcollis2]

As things stand we think the best workaround may be to enforce firewall on storage accounts which can be done with policy. This means that even if public is set, the firewall will block access. This is not fool proof (someone could add a 0.0.0.0/0 address to the firewall or something) but is somewhat preventative. The issue with this approach is that we would have to re-architect our solution to support vnet integrated storage accounts which is not a small task.

Sent from my iPhone

On Aug 15, 2018, at 06:25, retux notifications@github.com wrote:

IMHO it would be a great feature to add from a security standpoint. Looking forward for that. In the meantime, is there any workaround at least to audit blob containers with public permissions?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[eladperets]

@klawrie, @Yliche,

Unfortunately we only support resources managed by Azure Resource Manager (i.e. resource that can be managed by calling https://management.azure.com/[ResourceId]). It looks like the access polices are not modeled as ARM resource.

The only way I could find to access them are by calling directly to storage blob service, which we don't support.

Elad

[abhijay5]

@klawrie @mcollis2 Did the above policy worked for you guys as i am getting an error for alias doesn't exist

[abhijay5]

@mcollis2 Can you please share the policy for the Azure CIS Benchmark 3.7 (returns "blob" or "container" if access is enabled, or null if public access isn't enabled. )

[mcollis2]

Hi, the alias does not exist so we did not get the policy to work. The arm template syntax works for deploying containers in a template, but no alias is provided by the policy team to enforce this attribute. Furthermore my discussions with the policy team indicated that due to how storage apis work, that it is not possible to use a policy alias to enforce this as it would not apply to the still existing old api. My understanding is that storage team are working on a new functionality to prevent public access outright, and once that is provided then a new policy item will be available to enforce it.

Our workaround is to enable firewall and whitelist our physical location ip for now.

Thanks, Matthew

Sent from my iPhone

On Nov 13, 2018, at 03:36, abhijay5 notifications@github.com wrote:

@mcollis2 Can you please share the policy for the Azure CIS Benchmark 3.7 (returns "blob" or "container" if access is enabled, or null if public access isn't enabled. )

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[fhryo-msft]

Azure Storage is introducing a new feature to prevent public access on account level. It can be used to use Azure Policy to enforce disabling public access across storage accounts. The ETA is 2019 H2.

[mentat9]

We have published the full set of aliases for Microsoft.Storage. Is this the alias you need?

[C:\PS]-> Get-AzPolicyAlias -NamespaceMatch Microsoft.Storage -ResourceTypeMatch storageAccounts/blobServices | select -ExpandProperty
Aliases | ?{ $_.Name -like '*publicAccess*' }

Name                                                                           Paths
----                                                                           -----
Microsoft.Storage/storageAccounts/blobServices/default.containers.publicAccess {Microsoft.Azure.Management.ResourceManager.Models.AliasPathType}

[C:\PS]->

[fhryo-msft]

@klawrie can confirm. But i think the alias doesn't satisfy the audit scenario fully. The reason is that Azure Policy is evaluated when a change occurs to resource in management plane with ARM operations. However, public access on container is configurable in both ARM operations and data plane operations with Storage SDK. So when the access is changed from ARM operations, Azure Policy can catch and audit. If it's changed from data plane, Azure Policy can't capture.

The coming feature will be controlled as property in storage account. It will be fully managed by ARM operations. And when it's enabled, the access change will fail from both planes.

[techn0rome0]

Azure Storage is introducing a new feature to prevent public access on account level. It can be used to use Azure Policy to enforce disabling public access across storage accounts. The ETA is 2019 H2.

Any update on this? If its not possible has anyone had luck on reporting on this via log analytics?

[fhryo-msft]

The ETA is not changed. With the query of activity logs in Log Analytics, you can find logs when container access is changed through ARM operations. Reference: https://docs.microsoft.com/en-us/azure/azure-monitor/platform/collect-activity-logs.

But when container access is changed with Storage SDK, the query won't find as the operation will be tracked in Storage analytics logs. It will be supported when Storage on-boards data plane logs to Azure Monitor where can be ingested in Log Analytics just like Activity Logs.

[techn0rome0]

What is the appropriate query for this? When I change the storage status from front-end to/from public or private, I cannot find the logs anywhere in the AzureActivity table.

[fhryo-msft]

I am not sure what you mean by "from front-end". If you are using Set Container ACL (https://docs.microsoft.com/en-us/rest/api/storageservices/set-container-acl), the request is logged in storage analytics logs as it's data plane operation. If you are using Blob container update (https://docs.microsoft.com/en-us/rest/api/storagerp/blobcontainers/update), the request is logged in azure activity log as it's management plane operation. The query in Log Analytics looks like:

AzureActivity | where OperationNameValue contains "Microsoft.Storage/storageAccounts/blobServices/containers/write"

[fhryo-msft]

@klawrie , updates to your original ask. CLI to query containers with public access az storage container list --account-name sbuswestcentral --query "[?properties.publicAccess!=null]"

Alias used in Azure Policy Microsoft.Storage/storageAccounts/blobServices/containers/publicAccess

Command to find alias for container public access Get-AzPolicyAlias -NamespaceMatch Microsoft.Storage -ResourceTypeMatch storageAccounts/blobServices/containers | select -ExpandProperty Aliases

Sample Azure Policy to deny enabling container public access with ARM operations {   "if": {     "allOf": [       {         "field": "type",         "equals": "Microsoft.Storage/storageAccounts/blobServices/containers"       },       {         "not": {           "field":"Microsoft.Storage/storageAccounts/blobServices/containers/publicAccess",           "equals": "None"         }       }     ]   },   "then": {     "effect": "deny"   } }

Please note that Azure Policy won't be triggered if container public access is enabled with Storage SDK.

[techn0rome0]

Have we confirmed this works?

[fhryo-msft]

If you mean my last post, then yes. It works.

[mattyt0406]

I have tried the above policy and it did not work as the field is not the correct alias "field":"Microsoft.Storage/storageAccounts/blobServices/containers/publicAccess", = "field":"Microsoft.Storage/storageAccounts/blobServices/default.containers.publicAccess",

This still has not fired on any of our existing resources that have Blob/Container level set.
I have also change the type from: "equals": "Microsoft.Storage/storageAccounts/blobServices/containers" to: "equals": "Microsoft.Storage/storageAccounts/blobServices" to see if anything would fire....

Still no luck

[rjoncloud]

I agree with mattyt0406 . I tried to implement the policy, but it doesn't work.

[mattyt0406]

@rjoncloud - Are you in Gov or Commercial cloud?

[fhryo-msft]

@mattyt0406 , when you say it's not fired on any of existing resources, how do you test? Azure policy won't be triggered if there is no ARM operation occurred on resources. So you need to either create a container with container create API (https://docs.microsoft.com/en-us/rest/api/storagerp/blobcontainers/create) or update an existing container with container update API (https://docs.microsoft.com/en-us/rest/api/storagerp/blobcontainers/update)

[rjoncloud]

@mattyt0406 - I am in commercial cloud.

@fhryo-msft - I created a custom policy in Azure policies and assigned to a RG. Then I created as well as changed permission in the storage account containers through Azure portal. It allowed to change the permissions and also allowed creation of a new container with public access. I waited for more than 30 min for the policy to take effect but no luck.

[fhryo-msft]

@rjoncloud , i feel like to schedule a meeting with you for this. What's best way to reach you?

[mattyt0406]

Ok to give an update...we worked with @mentat9 to iron out the details on the Gov side. It looked like it fired for one when we had the Audit effect but since we have gone to a deny it appears our compliance has gone to 100% and is not firing on new containers. Strange thing is we see the Denys being tried in the activity log but the containers /w access policy of X both still get built. @fhryo-msft if you want to meet my alias is mataylo. we currently hve a sevA/Cristsit opened on this. :(

[rjoncloud]

I am also planning to open a support ticket. Hopefully we can get some help. Will try audit mode.

[techn0rome0]

I am still having trouble getting this to work. I am still able to create blobs and set them under any status without a deny.

[techn0rome0]

@rjoncloud , i feel like to schedule a meeting with you for this. What's best way to reach you? Id like to be on the call.

[dani3lheidemann]

Hi all, any updates on this? We also urgently need the Azure Policy Alias for "Microsoft.Storage/storageAccounts/blobServices/containers" to deny programmatically public containers.

Furthermore I have the same problem like @techn0rome0 that there's nothing in the activity logs that indicates that a container was changed from private to public access.

Thank you all, Daniel

[mentat9]

Please see the Known Issues section of the readme for information and current status on this issue.

[rroman81]

@mentat9 is there an ETA on when storage account team is planning to finish implementing the blog operations via storage management API? Is there a tech community item that we can track? Clients will greatly benefit from this. Also, having a workaround how to detect public blobs through automation would be helpful to spell out while everyone is waiting.

[fhryo-msft]

@rroman81 , Storage team will release the control on storage account level in Q3. We will have announcement for this.

In the meanwhile, you can setup alert for monitoring anonymous access based on Storage metrics. You can use Transactions with dimension Authentication equal to Anonymous to configure your alert rule. You can find reference to the metrics in: https://docs.microsoft.com/en-us/azure/storage/common/storage-metrics-in-azure-monitor#transaction-metrics

[rroman81]

@fhryo-msft thank you for speedy reply. I'll steer the client towards monitoring workaround for now. Thanks.

[dani3lheidemann]

@fhryo-msft I just checked your specified workaround. This works for me, but if this workaround detects an anonymous access, the access level of the storage account has already been changed and used - so in my opinion it's "too late" because your private data has already been compromised. To "copy" this "deny"-effect from Azure Policy for this use case it may make sense to use Azure Automation to monitor and correct proactively wrong access level containers. You could check this with PowerShell Snippet like

Get-AzStorageAccount -Name xy -ResourceGroupName xy | Get-AzStorageContainer and correct the access level automatically.

Or, you can create a Watcher Task in Azure Automation to monitor access level of Storage Accounts and a second runbook for access level correction?

What do you think about this, @fhryo-msft ? Thanks for your feedback! Best, Daniel

[rroman81]

@nnamedieh I do agree that using automation is the way to go as a workaround. I am heading that route myself. The monitoring solution is only helpful for alerting purposes and not for enforcement. Also, consider the work done by AzSK as well. I am still exploring that avenue as well to help automate security scanning.

[fhryo-msft]

@nnamedieh , your opinion is reasonable and the objective of above workaround is to stop bleeding when the risk becomes real issue. If the scale is not an issue in your case, your approach should work properly. When you have large number of containers and large number of storage accounts, enumerating storage accounts and containers may not succeed completely and consistently. In the meanwhile, keeping listing containers is also expensive operation which is billed $0.05 per 10 K operations (Reference: https://azure.microsoft.com/en-us/pricing/details/storage/blobs/)

Literally enforcing it on account level with Azure Policy would be the most effective solution to avoid dealing with various of limits on response time, scale, cost.

[rjoncloud]

Is this working anymore? I am getting an error for the alias:

The resource type 'storageAccounts/blobServices/containers' referenced by the 'field' property 'Microsoft.Storage/storageAccounts/blobServices/containers/publicAccess' of the policy rule doesn't exist under provider 'Microsoft.Storage'.

[mentat9]

Please see the Known Issues section in the readme for the latest information on this resource type.

[mcollis2]

Just wanted to update that you can now prevent public access on storage containers through policy as of July 2020.

https://docs.microsoft.com/en-us/azure/storage/blobs/anonymous-read-access-prevent

Here is a policy that facilitates enforcing this: https://github.com/davidokeyode/charis-cloud-azure-policy/blob/master/storage/storage-account-public-blob/azurepolicy.json

[fhryo-msft]

This property is optional in storage account. By default, it doesn't exist until it's configured explicitly.

What result will the following logic be evaluated as when allowBlobPublicAccess doesn't exist in this case?

       {
          "field": "Microsoft.Storage/storageAccounts/allowBlobPublicAccess",
          "notEquals": "false"
        }