In reference to ticket #1237 Duende IdentityServer6 upgrade - backward compatibility - token introspect issue- token created with IdentityServer4 is not work after upgrade. #1237

[susan12-web]

Which version of Duende IdentityServer are you using? 6.3.3

Which version of .NET are you using? .net 6

Describe the bug

A clear and concise description of what the bug is. --- Token created with IdentityServer4 was supplied to relying party this was having 1 year time. So after the code upgrade from idp 4 to duende idp 6 all these grant data inside refresh tokens are failing to deserialize.

To Reproduce

https://localhost:5001/connect/external/introspect during the api call to external introspect. below lines for code will internal call persistent grants which is failing to deserialze :

public async Task GetRefreshTokenAsync(string refreshTokenHandle) { var obRefreshToken = await GetItemAsync(refreshTokenHandle); return obRefreshToken; }

protected virtual async Task GetItemAsync(string key) { var hashedKey = GetHashedKey(key); var item = await GetItemByHashedKeyAsync(hashedKey); if (item == null) { Logger.LogDebug("{grantType} grant with value: {key} not found in store.", GrantType, key); } return item; }

var grant = await Store.GetAsync(hashedKey); if (grant != null && grant.Type == GrantType) { try { var data = grant.Data; return Serializer.Deserialize(output); ---- > failing here with error Message=The JSON value could not be converted to Duende.IdentityServer.Models.AccessTokenType. Path: $.AccessToken.AccessTokenType | LineNumber: 140 | BytePositionInLine: 34.

} catch (Exception ex) { Logger.LogError(ex, "Failed to deserialize JSON from grant store."); } Expected behavior

Expecting some way for duende to support old tokens generated along with new.

Log output/exception with stacktrace

System.Text.Json.JsonException HResult=0x80131500 Message=The JSON value could not be converted to Duende.IdentityServer.Models.AccessTokenType. Path: $.AccessToken.AccessTokenType | LineNumber: 140 | BytePositionInLine: 34. Source=System.Text.Json StackTrace: at System.Text.Json.ThrowHelper.ThrowJsonException(String message) at System.Text.Json.Serialization.Converters.EnumConverter1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) at System.Text.Json.Serialization.Metadata.JsonPropertyInfo1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader) at System.Text.Json.Serialization.Converters.ObjectDefaultConverter1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) at System.Text.Json.Serialization.JsonConverter1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) at System.Text.Json.Serialization.Metadata.JsonPropertyInfo1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader) at System.Text.Json.Serialization.Converters.ObjectDefaultConverter1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) at System.Text.Json.Serialization.JsonConverter1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) at System.Text.Json.Serialization.JsonConverter1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable1 actualByteCount) at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo) at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options) at Duende.IdentityServer.Stores.Serialization.PersistentGrantSerializer.Deserialize[T](String json) at IdentityProvider.Module.Stores.OpenBankingRefreshTokenStore.d__8.MoveNext() in C:\dev\projects\openbanking\digitalidentityprovider\src\app\IdentityProvider.Module\src\library\IdentityProvider.Module\Stores\OpenBankingRefreshTokenStore.cs:line 116

This exception was originally thrown at this call stack: [External Code] IdentityProvider.Module.Stores.OpenBankingRefreshTokenStore.GetItemByHashedKeyAsync(string) in OpenBankingRefreshTokenStore.cs

data--- refresh token Additional context

Add any other context about the problem here.

Identity server4 version -4.1.2-- is the existing older version and the token is generated from 4 and trying to deserialize using idp 6

sample json of idp4 grant data that is been serializing :

{ "OriginalSubject": { "AuthenticationType": "IdentityServer4", "Claims": [{ "Type": "sub", "Value": "74337624", "ValueType": "http://www.w3.org/2001/XMLSchema#string" } ] }, "TokenHandle": "E9B809E256F2A966068443B962079B24567D4883CF5A63D00B52C5B131A6D4FT", "CreationTime": "2023-09-11T23:50:20Z", "Lifetime": 31535999, "ConsumedTime": null, "AccessToken": { "AllowedSigningAlgorithms": [], "Confirmation": null, "Audiences": ["bank:accounts.basic:read", "bank:accounts.detail:read", "bank:payees:read", "bank:regular_payments:read", "bank:transactions:read", "common:customer.basic:read", "common:customer.detail:read", "https://kong-proxy-ingress.apig-test.obn02.digobp01.aws.test.au.internal.cba/infosec/resources"], "Issuer": "https://kong-proxy-ingress.apig-test.obn02.digobp01.aws.test.au.internal.cba/infosec", "CreationTime": "2023-09-11T23:50:20Z", "Lifetime": 1296000, "Type": "access_token", "ClientId": "5NsqddxhU2m5lEugWCjVZOfl2ECM8XPyqS54aNLp8II", "AccessTokenType": "Reference", "Description": "_OfflineAccessScopeManuallyAdded", "Claims": [{ "Type": "client_id", "Value": "5NsqddxhU2m5lEugWCjVZOfl2ECM0XPyqS54aNLp8II", "ValueType": "http://www.w3.org/2001/XMLSchema#string" } ], "Version": 4 }, "Version": 4 }

Another issue is the below one :

Failed to deserialize JSON from grant store.","Exception":"System.Text.Json.JsonException: The JSON value could not be converted to Duende.IdentityServer.Models.AccessTokenType

@AndersAbel -- Sorry for the late reply.

It is also a bit confusing if you are really working with refresh or reference tokens. In the stack trace there is a method called OpenBankingRefreshTokenStore. The Json however has a token type of "Reference". Do you know which one it actually is that you are trying to use?

Am working with both refresh and reference token grant types.(for internal and external introspection). The accesstokentype is creating the problem with idp 6. For each type for older version I have to do the below by overriding DefaultGrantStore,

GetItemByHashedKeyAsync method as below :

protected override async Task GetItemByHashedKeyAsync(string hashedKey) { var grant = await Store.GetAsync(hashedKey); if (grant != null && grant.Type == GrantType) { try { var data = grant.Data; dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(data); if (jsonObj != null && GrantType == PersistedGrantTypes.RefreshToken && jsonObj["OriginalSubject"]["AuthenticationType"] == Common.Constants.AuthenticationTypeidpserv4) { jsonObj["AccessToken"]["AccessTokenType"] = Convert.ToInt32(AccessTokenType.Reference); } else if (jsonObj != null && GrantType == PersistedGrantTypes.ReferenceToken && jsonObj["Type"] == TokenTypes.AccessToken) { if (jsonObj["Version"] == Common.Constants.Version) jsonObj["AccessTokenType"] = Convert.ToInt32(AccessTokenType.Reference); } string output = Newtonsoft.Json.JsonConvert.SerializeObject(jsonObj, Newtonsoft.Json.Formatting.Indented); return Serializer.Deserialize(output); } catch (Exception ex) { Logger.LogError(ex, "Failed to deserialize JSON from grant store."); } }

return default; }

Is there any other way to handle this?

[RolandGuijt]

I've added a link to this issue to the original issue which I reopened. Please track the issue from there since I'm closing this one.