Open apopa92 opened 4 years ago
We have a very similar setup and we are experiencing similar problems.
@apopa92 I think I have got to the bottom of this. The problem is that the access token is expiring (they appear to be valid for one hour) so the next time the DP mechanism tries to refresh the keys, it's using an expired token.
You can fix this by using an overload of the TokenCredential
constructor that enables periodic refresh, and implement a method to perform the refresh.
var storageTokenCredential = new TokenCredential(
accessToken,
PeriodicStorageTokenRenewer,
null,
TimeSpan.FromMinutes(27));
// ...
private async Task<NewTokenAndFrequency> PeriodicStorageTokenRenewer(
object state,
CancellationToken cancellationToken)
{
var accessToken = await GetAccessTokenAsync("https://storage.azure.com/");
var tokenAndFrequency = new NewTokenAndFrequency { Token = accessToken };
return tokenAndFrequency;
}
See https://github.com/dotnet/aspnetcore/issues/9012 for some background.
We are having the same issue and we are not using managed identities, we are using a SAS token generated with the storage account key that is valid for one hour. The issue doesn't occur until about 15 hours after deployment time.
public static IServiceCollection AddWebDataProtection(this IServiceCollection services, ResultsAndCertificationConfiguration config, IWebHostEnvironment env)
{
services.AddDataProtection()
.PersistKeysToAzureBlobStorage(GetDataProtectionBlobTokenUri(config));
return services;
}
private static Uri GetDataProtectionBlobTokenUri(ResultsAndCertificationConfiguration config)
{
var cloudStorageAccount = new CloudStorageAccount(new StorageCredentials(config.BlobStorageSettings.AccountName, config.BlobStorageSettings.AccountKey), useHttps: true);
var blobClient = cloudStorageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference(config.DataProtectionSettings.ContainerName);
var blob = container.GetBlockBlobReference(config.DataProtectionSettings.BlobName);
var sharedAccessPolicy = new SharedAccessBlobPolicy()
{
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(1),
Permissions = SharedAccessBlobPermissions.Read | SharedAccessBlobPermissions.Write | SharedAccessBlobPermissions.Create
};
var sasToken = blob.GetSharedAccessSignature(sharedAccessPolicy);
return new Uri($"{blob.Uri}{sasToken}");
}
Hi,
We have a website that sits on a webfarm (3 servers - Azure VMs behind load balancer).
The VMs have Managed Identities enabled and we are using the following code to enable the DataProtection, store the keys in the AzureStorage and encrypt them with the help of AzureKeyVault.
When the website starts, it creates the data protection key in the AzureStorage and everything seems to be working without any issues. The errors start pouring in after around 12-16 hours and they keep going, sometimes for a couple of hours, somtimes for a tens of hours.