[BUG] Accessing blob container fails with auth error, when using Azure Front Door CDN

[StannieV]

Library name and version

Azure.Storage.Blobs 12.15.0

Describe the bug

When creating a BlobServiceClient based on an Azure Front Door endpoint URL, in ~50% of the cases, it produces an Authentication exception.

Expected behavior

In my solution, I want to create a SAS Uri for a blob item that is accessed via an Azure Front Door (AFD) endpoint. The blob container cannot be publicly accessible and therefore the SAS is needed. After creating a SAS for a blob item, you can replace the hostname of the blob URL E.g ‘https://.blob.core.windows.net’ (from the complete SAS) with the hostname of the AFD endpoint. E.g: ‘https:// .z01.azurefd.net’. With this new URL, you can access the blob item (even download it in the browser) through the AFD endpoint and use CDN.

In code, it seemed a clumsy solution first to create a SAS and then replace the hostname of the URL in the code. After some searching, I found a solution in creating a ‘BlobServiceClient’ with a different service URL, which points to the AFD endpoint. At first glance, this seems great. With this, I only have to change the initialization of the blob object(s) in my startup logic and keep my logic in the application the same. Unfortunately, this is not always working. In half of the cases, it works and in the other half, an exception is thrown in the code.

Why is this exception thrown and should it work as I intend? Or, are there better alternatives for what I want to achieve?

Actual behavior

The exception that is produced is:

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:abba20e0-f01e-004d-1969-52f96d000000
Time:2023-03-09T09:26:42.6689043Z
Status: 403 (Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.)
ErrorCode: AuthenticationFailed

Additional Information:
AuthenticationErrorDetail: The MAC signature found in the HTTP request 'C7MEyGoAuJVMoVG7whS58c46CW/2JUlPEiQoylCt4ok=' is not the same as any computed signature. Server used following string to sign: 'GET

bytes=0-8388607
x-ms-client-request-id:9798c48e-0480-4e8e-990e-2a5f615ec103
x-ms-date:Thu, 09 Mar 2023 09:26:42 GMT
x-ms-return-client-request-id:true
x-ms-version:2021-12-02
/<NAME>/machineimages
comp:list
restype:container'.

Content:
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:abba20e0-f01e-004d-1969-52f96d000000
Time:2023-03-09T09:26:42.6689043Z</Message><AuthenticationErrorDetail>The MAC signature found in the HTTP request 'C7MEyGoAuJVMoVG7whS58c46CW/2JUlPEiQoylCt4ok=' is not the same as any computed signature. Server used following string to sign: 'GET

bytes=0-8388607
x-ms-client-request-id:9798c48e-0480-4e8e-990e-2a5f615ec103
x-ms-date:Thu, 09 Mar 2023 09:26:42 GMT
x-ms-return-client-request-id:true
x-ms-version:2021-12-02
/<NAME>/machineimages
comp:list
restype:container'.</AuthenticationErrorDetail></Error>

Headers:
Date: Thu, 09 Mar 2023 09:26:42 GMT
Connection: keep-alive
x-ms-request-id: abba20e0-f01e-004d-1969-52f96d000000
x-ms-error-code: AuthenticationFailed
Access-Control-Expose-Headers: REDACTED
Access-Control-Allow-Origin: *
x-azure-ref: REDACTED
X-Cache: REDACTED
Content-Type: application/xml
Content-Length: 800

Reproduction Steps

The following C# code reproduces (sometimes) the exception:

using System;
using System.Threading.Tasks;
using Azure.Storage;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;

namespace ConsoleApp23
{
 internal class Program
 {
 static async Task Main(string[] args)
 {
 try
 {
 //Note1: Add a random image to the 'machineimages' container before running the code

 var accountName = "mystoragetst01";
 var accountKey = "*****************************************************************";

 //Note3: This is the original blob url that works without problems
 //string serviceUrl = "https://mystoragetst01.blob.core.windows.net";

 //Note4: This is the Azure Front Door url (Update the after running the BICEP script!) which throw regularly (~50%) an exception when using this.
 string serviceUrl = "https://mystoragetst01-?????????.z01.azurefd.net"; //TODO: Update url based on AFD endpoint after running the BICEP script!

 string containerName = "machineimages";

 StorageSharedKeyCredential credential = new StorageSharedKeyCredential(accountName, accountKey);
 BlobServiceClient blobServiceClient = new BlobServiceClient(new Uri(serviceUrl), credential);
 BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);

 await foreach (BlobItem blobItem in containerClient.GetBlobsAsync()) //--> This line will produce the exception: 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.'
 {
 Console.WriteLine(blobItem.Name);
 }
 }
 catch (Exception exception)
 {
 Console.WriteLine(exception);
 }
 }
 }
}

Project file:

<Project Sdk="Microsoft.NET.Sdk">

 <PropertyGroup>
 <OutputType>Exe</OutputType>
 <TargetFramework>net7.0</TargetFramework>
 </PropertyGroup>

 <ItemGroup>
 <PackageReference Include="Azure.Storage.Blobs" Version="12.15.0" />
 </ItemGroup>

</Project>

The following BICEP script creates the Storage Account and the Azure Front Door (I composed it from multiple files so it isn’t production code):

param location string = resourceGroup().location

var storageAccountName = 'mystoragetst01'

resource storageAccountResource 'Microsoft.Storage/storageAccounts@2022-09-01' = {
 name: storageAccountName
 location: location
 kind: 'StorageV2'
 sku: {
 name: 'Standard_LRS'
 }
 properties: {
 minimumTlsVersion: 'TLS1_2'
 allowBlobPublicAccess: false
 allowSharedKeyAccess: true
 supportsHttpsTrafficOnly: true
 accessTier: 'Hot'
 }
}

//CORS rules
var corsProperty = {
 corsRules:[
 {
 allowedOrigins: [ 
 '*' 
 ]
 allowedHeaders: [
 '*'
 ]
 allowedMethods: [
 'GET'
 ]
 maxAgeInSeconds: 0
 exposedHeaders: [
 '*'
 ]
 }
 ]
}

//Set Data protection/Enable soft delete for blobs
resource blobServicesResource 'Microsoft.Storage/storageAccounts/blobServices@2022-09-01' = {
 parent: storageAccountResource
 name: 'default'
 properties: {
 containerDeleteRetentionPolicy: {
 enabled: true
 days: 14
 }
 deleteRetentionPolicy: {
 enabled: true
 days: 14
 }
 cors: corsProperty
 }
}

//Create container: 
resource containerResource 'Microsoft.Storage/storageAccounts/blobServices/containers@2022-09-01' = {
 name: '${storageAccountName}/default/machineimages'
 dependsOn: [

 ]
}

//Set Lifecycle management
resource storageAccountName_default 'Microsoft.Storage/storageAccounts/managementPolicies@2022-09-01' = {
 name: '${storageAccountName}/default'
 dependsOn: [
 storageAccountResource
 ]
 properties: {
 policy: {
 rules: [
 {
 enabled: true
 name: 'DeleteBlobInMessagesAfter186Days'
 type: 'Lifecycle'
 definition: {
 actions: {
 baseBlob: {
 delete: {
 daysAfterModificationGreaterThan: 186
 }
 }
 }
 filters: {
 blobTypes: [
 'blockBlob'
 ]
 prefixMatch: [
 'messages'
 ]
 }
 }
 }
 ]
 }
 }
}

//Create Azure Front Door
var frontDoorName = 'MyAFDTest01'

resource azureFrontDoorResource 'Microsoft.Cdn/profiles@2022-11-01-preview' = {
 name: frontDoorName
 location: 'Global'
 sku: {
 name: 'Premium_AzureFrontDoor'
 }
 properties: {
 originResponseTimeoutSeconds: 60
 extendedProperties: {
 }
 }
}

//Retrieves host name by substring 'https://mystoragetst01.blob.core.windows.net/' into 'mystoragetst01.blob.core.windows.net'
var storageAccountHostName = substring(storageAccountResource.properties.primaryEndpoints.blob, 8, length(storageAccountResource.properties.primaryEndpoints.blob) - 8 - 1)

//Add Endpoint to Azure Front Door
var endpointName = storageAccountName
var originGroupName = 'MyStorageTstOriginGroup'
var originName = 'MyStorageTstOrigin'
var originHostName = storageAccountHostName
var endpointRouteName = 'MyStorageTstRoute'
var webApplicationFirewallPolicyName = 'MyWafTest01'
var webApplicationFirewallPolicyMode = 'Detection'
var securityPolicyName = 'MyStorageTstSecurityPolicy'

//AFD Endpoint
resource endpointResource 'Microsoft.Cdn/profiles/afdendpoints@2022-11-01-preview' = {
 parent: azureFrontDoorResource
 name: toLower(endpointName)
 location: 'Global'
 properties: {
 enabledState: 'Enabled'
 }
}

//ADF Origin Group
resource originGroupResource 'Microsoft.Cdn/profiles/origingroups@2022-11-01-preview' = {
 parent: azureFrontDoorResource
 name: originGroupName
 properties: {
 loadBalancingSettings: {
 sampleSize: 4
 successfulSamplesRequired: 3
 additionalLatencyInMilliseconds: 50
 }
 healthProbeSettings: {
 probePath: '/'
 probeRequestType: 'HEAD'
 probeProtocol: 'Http'
 probeIntervalInSeconds: 100
 }
 sessionAffinityState: 'Disabled'
 }
}

//ADF Origin
resource originResource 'Microsoft.Cdn/profiles/origingroups/origins@2022-11-01-preview' = {
 parent: originGroupResource
 name: originName
 properties: {
 hostName: toLower(originHostName)
 httpPort: 80
 httpsPort: 443
 originHostHeader: toLower(originHostName)
 priority: 1
 weight: 1000
 enabledState: 'Enabled'
 enforceCertificateNameCheck: true
 }
}

//ADF Endpoint Route
resource routeResource 'Microsoft.Cdn/profiles/afdendpoints/routes@2022-11-01-preview' = {
 parent: endpointResource
 name: endpointRouteName
 properties: {
 cacheConfiguration: {
 compressionSettings: {
 isCompressionEnabled: true
 contentTypesToCompress: [
 'application/eot'
 'application/font'
 'application/font-sfnt'
 'application/javascript'
 'application/json'
 'application/opentype'
 'application/otf'
 'application/pkcs7-mime'
 'application/truetype'
 'application/ttf'
 'application/vnd.ms-fontobject'
 'application/xhtml+xml'
 'application/xml'
 'application/xml+rss'
 'application/x-font-opentype'
 'application/x-font-truetype'
 'application/x-font-ttf'
 'application/x-httpd-cgi'
 'application/x-javascript'
 'application/x-mpegurl'
 'application/x-opentype'
 'application/x-otf'
 'application/x-perl'
 'application/x-ttf'
 'font/eot'
 'font/ttf'
 'font/otf'
 'font/opentype'
 'image/svg+xml'
 'text/css'
 'text/csv'
 'text/html'
 'text/javascript'
 'text/js'
 'text/plain'
 'text/richtext'
 'text/tab-separated-values'
 'text/xml'
 'text/x-script'
 'text/x-component'
 'text/x-java-source'
 ]
 }
 queryStringCachingBehavior: 'IgnoreQueryString'
 }
 customDomains: []
 originGroup: {
 id: originGroupResource.id
 }
 ruleSets: []
 supportedProtocols: [
 'Https'
 ]
 patternsToMatch: [
 '/*'
 ]
 forwardingProtocol: 'HttpsOnly'
 linkToDefaultDomain: 'Enabled'
 httpsRedirect: 'Disabled'
 enabledState: 'Enabled'
 }
}

//Create Web Application Firewall
resource frontdoorWebApplicationFirewallPolicyResource 'Microsoft.Network/frontdoorwebapplicationfirewallpolicies@2022-05-01' = {
 name: webApplicationFirewallPolicyName
 location: 'Global'
 sku: {
 name: 'Premium_AzureFrontDoor'
 }
 properties: {
 policySettings: {
 enabledState: 'Enabled'
 mode: webApplicationFirewallPolicyMode
 customBlockResponseStatusCode: 403
 requestBodyCheck: 'Enabled'
 }
 customRules: {
 rules: []
 }
 managedRules: {
 managedRuleSets: [
 {
 ruleSetType: 'Microsoft_DefaultRuleSet'
 ruleSetVersion: '2.0'
 ruleSetAction: 'Block'
 ruleGroupOverrides: []
 exclusions: []
 }
 {
 ruleSetType: 'Microsoft_BotManagerRuleSet'
 ruleSetVersion: '1.0'
 ruleGroupOverrides: []
 exclusions: []
 }
 ]
 }
 }
}

//Web Application Firewall connection to the Endpoint
resource securityPolicyResource 'Microsoft.Cdn/profiles/securitypolicies@2022-11-01-preview' = {
 parent: azureFrontDoorResource
 name: securityPolicyName
 properties: {
 parameters: {
 wafPolicy: {
 id: frontdoorWebApplicationFirewallPolicyResource.id
 }
 associations: [
 {
 domains: [
 {
 id: endpointResource.id
 }
 ]
 patternsToMatch: [
 '/*'
 ]
 }
 ]
 type: 'WebApplicationFirewall'
 }
 }
}

Environment

NET runtime version: .Net 6.0 & .Net 7.0 IDE and version: Microsoft Visual Studio Enterprise 2022 (64-bit) - Current Version 17.5.1 OS: Windows 11 Pro Version 22H2 (OS Build 22621.1344)

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[navba-MSFT]

@StannieV Apologies for the late reply. Thanks for reaching out to us and reporting this issue. We are looking into this issue and we will provide an update.

[navba-MSFT]

@StannieV Could you please share the x-ms-request-id from the most recent occurrence of the issue ?

[StannieV]

Hi @navba-MSFT ,

Thanks for looking at my issue. The following request ids are from failing ones (all from a few minutes ago):

1. x-ms-request-id: c511d5b4-101e-0045-1f96-55e362000000
2. x-ms-request-id: 6fe774f3-601e-0070-1d97-558f76000000
3. x-ms-request-id: 6fe7a3fe-601e-0070-5297-558f76000000
4. x-ms-request-id: 6fe806f5-601e-0070-4397-558f76000000
5. x-ms-request-id: 6fe8451d-601e-0070-7f97-558f76000000

I hope this is enough!

[navba-MSFT]

@StannieV Thanks for sharing the request ID. I checked the backend logs and the failure is due to the StringToSign. As mentioned in this documentation, You should first create the successfully working SAS token with Blob (storage) endpoint. Then use the SAS token query string to append it to your frontdoor endpoint url. Hope this helps.

[StannieV]

Hi @navba-MSFT,

Thanks for investigating the requests. I had hoped for a more elegant way to solve this in code than string manipulation.

Based on your answer I still have some questions:

1. Why does the authentication error sometimes occur and sometimes not? This seems inconsistent behavior.
2. So the service URL in the constructor, of the 'BlobServiceClient' object, cannot be used in combination with Azure Front Door?
3. When do can you use the service URL in the constructor of the 'BlobServiceClient' object?

[navba-MSFT]

@StannieV Thanks for your reply. While I tested at my end using your above code, I could see that it was using Sharedkey and not SAS token. So I had to update your code to make use of SAS token with the Azure FrontEnd endpoint and that worked just fine. Please refer the sample code here. Hope this helps.

[StannieV]

@navba-MSFT you're right, my example code doesn't generate a SAS token. I simplified my program for this issue. But the problem also occurs in my example code on retrieving the blob items. (containerClient.GetBlobsAsync()) If this succeeds, generating the SA token will also work. It will look like this:

// Get a reference to the container client
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);

// Use the container client to perform blob operations
// For example, you can list the blobs in the container:

await foreach (BlobItem blobItem in containerClient.GetBlobsAsync())
{
    Console.WriteLine(blobItem.Name);
    var blobClient = containerClient.GetBlobClient(blobItem.Name);

    var blobSasBuilder = new BlobSasBuilder();

    blobSasBuilder.SetPermissions(BlobSasPermissions.Read);
    blobSasBuilder.StartsOn = DateTimeOffset.UtcNow.AddMinutes(-5);
    blobSasBuilder.ExpiresOn = DateTimeOffset.UtcNow.Add(TimeSpan.FromHours(5));

    blobSasBuilder.BlobContainerName = containerClient.Name;
    blobSasBuilder.BlobName = blobClient.Name;

    var sas = blobClient.GenerateSasUri(blobSasBuilder);

    Console.WriteLine(sas);
}

[navba-MSFT]

@StannieV Thanks for your reply. Let's isolate this issue first.

1. Do you want to create an Account SAS ? or Container SAS ? or a Blob SAS ?
2. The sample which I used above is an account SAS. So before I test your SAS generating code, Could you please confirm if hardcoding the SAS token as mentioned in my sample code here, works fine at your end ? Awaiting your reply.

[navba-MSFT]

@StannieV I wanted to do quick follow-up to check if you had a chance to look at my above comment. Please let us know if you had any updates on this. Awaiting your reply.

[StannieV]

Hi @navba-MSFT, I think I'll have today time to look at it.

[StannieV]

@StannieV Thanks for your reply. Let's isolate this issue first.

1. Do you want to create an Account SAS ? or Container SAS ? or a Blob SAS ?
2. The sample which I used above is an account SAS. So before I test your SAS generating code, Could you please confirm if hardcoding the SAS token as mentioned in my sample code here, works fine at your end ? Awaiting your reply.

@navba-MSFT

1. I want to create an blob SAS
2. The provided code sample does not work at my end. When I use 'https://?????.blob.core.windows.net' as serviceUri it works. But when using the Azure Front Door URL I get:

Response x-ms-client-request-id 'e6fabfca-cc37-43a2-b48e-a5adc9d2054f' does not match the original expected request id, 'b5bc9dd3-c086-41be-a7e8-87d8e08e3381'.
Unhandled exception. Azure.RequestFailedException: Response x-ms-client-request-id 'e6fabfca-cc37-43a2-b48e-a5adc9d2054f' does not match the original expected request id, 'b5bc9dd3-c086-41be-a7e8-87d8e08e3381'.

[navba-MSFT]

@StannieV Thanks for your reply. If hardcoding the SAS token itself is failing then that issue should be fixed first.

I tried creating a Blob SAS and used the same in my code to get that blob and it worked fine with FrontDoor URL. Sending you the screenshot from the fiddler if that helps:

Please check if your Frontdoor setting is configured as below:

[navba-MSFT]

@StannieV I wanted to do quick follow-up to check if you had a chance to look at my above comment. Please let us know if you had any updates on this. Awaiting your reply.

[StannieV]

@navba-MSFT I didn't have time to look at it. Most probably I'll have time next week... I'll keep you up-to-date.

[github-actions[bot]]

Hi @StannieV. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[github-actions[bot]]

Hi @StannieV, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!