search

DuendeSoftware / Support

Support for Duende Software products
20 stars 0 forks source link

Multiple versions of Duende.IdentityServer running at the same time #1296

Closed nunofilipecosta closed 1 week ago

nunofilipecosta commented 1 month ago

Which version of Duende IdentityServer are you using? 6 --> 7 Which version of .NET are you using? 6 --> 8 Describe the bug We have Duende.IdentityServer 6 ( dotnet 6 ) running in k8 with 3 pods. We did a canary deployment with 1 pod running Duende.IdentityServer 7 ( dotnet 8 ) and we started seeing errors of System.Security.Cryptography.CryptographicException on both versions We noticed that the database keys table now has 2 records and /.well-known/openid-configuration/jwks also return 2 keys and they get cached.

To Reproduce

Have 2 versions of .NET/Duende runnning at the same time ?

Expected behavior

No errors are expected

Log output/exception with stacktrace

Error unprotecting the IdentityServer signing key with kid "1FA073E3B68C05E32A79EBF7D9B79184". This is likely due to the ASP.NET Core data protection key that was used to protect it is not available. This could occur because data protection has not been configured properly for your load balanced environment, or the IdentityServer signing key store was populated with keys from a different environment with different ASP.NET Core data protection keys. Once you have corrected the problem and if you keep getting this error then it is safe to delete the specific IdentityServer signing key with that kid.

System.Security.Cryptography.CryptographicException: The payload was invalid. For more information go to https://aka.ms/aspnet/dataprotectionwarning
   at Microsoft.AspNetCore.DataProtection.Managed.ManagedAuthenticatedEncryptor.Decrypt(ArraySegment`1 protectedPayload, ArraySegment`1 additionalAuthenticatedData)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.DataProtection.DataProtectionCommonExtensions.Unprotect(IDataProtector protector, String protectedData)
   at Duende.IdentityServer.Services.KeyManagement.DataProtectionKeyProtector.Unprotect(SerializedKey key) in /_/src/IdentityServer/Services/Default/KeyManagement/DataProtectionKeyProtector.cs:line 56
   at Duende.IdentityServer.Services.KeyManagement.KeyManager.<GetAllKeysFromStoreAsync>b__20_0(SerializedKey x) in /_/src/IdentityServer/Services/Default/KeyManagement/KeyManager.cs:line 435

Additional context

Add any other context about the problem here.

AndersAbel commented 4 weeks ago

If I understand this right, you want to run two versions of IdentityServer in parallell in different pods. To make this succeed you would need to ensure that all preserved state is completely compatible between the two versions. To start with the IdentityServer operational store has changes to the database structure between these versions. Those changes may be backwards compatible if you are not using the new features, but that is something that would require some careful investigation. There is also other state that is written to cookies (the session cookies, the OIDC state cookies if integrating with upstream providers etc.) that might not be compatible beteween the .NET versions. Once more, careful investigation would be needed to know that everything is indeed compatible.

I would not recommend running two versions in parallell, there are just too many thing that need to be taken into consideration.

The error you have run into is related to Asp.Net Core Data Protection. We have a guide on data protection configuration that is a good starting point. If you had a working data protection setup in your v6 deployment I guess that it might be the application name that is not set. If the path for the v7 deployment is different then it will generate a different automatic name. To keep access to the existing data you would have to set the application name to the path of the v6 deployment.

Even if you don't try to run the two versions i parallell it is still important that the data protection settings are compatible to not lose access to existing date.

RolandGuijt commented 2 weeks ago

@nunofilipecosta Do you have anything to add to this issue? If not I'd like to close it.

nunofilipecosta commented 1 week ago

Hi !

@AndersAbel thank you for the comment. Our issue was indeed related to the Asp.Net Core Data Protection and the application name .

@RolandGuijt you can close the issue.

Thank you both!