Missing user attributes in refresh token response's ID token if Application's Token Issuer is JWT

Describe the issue: In a federated authentication scenario, we're relying in the hashed access token value for retrieving cached user attributes persisted in the session data store to populate the ID token claims [1].

Although this works fine for opaque tokens, we're not storing the actual hashed token in the case of JWT access tokens. Instead, we're actually storing the hashed 'jti' (JWT ID) value. Therefore, the database lookup [2][3] fails.

We must first get the token type from the OAuth Application Information, and substitute the access token value with the 'jti' field value whenever applicable. Example:

OAuthAppDO oAuthAppDO;
String accessToken;

try {
    oAuthAppDO = OAuth2Util.getAppInformationByClientId(clientId);
} catch (InvalidOAuthClientException e) {
    String error = "Error occurred while getting app information for client_id: " + clientId;
    throw new IdentityOAuth2Exception(error, e);
}

if (Objects.equals(oAuthAppDO.getTokenType(), "JWT")) {
    try {
        final JWT jwt = JWTParser.parse(accessToken);
        accessToken = jwt.getJWTClaimsSet().getJWTID();
    } catch (ParseException e) {
        log.error("Got a JWT access token but failed to obtain its 'jti' value. This could lead to missing data.", e);
        accessToken = tokenRespDTO.getAccessToken();
    }
} else {
    accessToken = tokenRespDTO.getAccessToken();
}

The affected code flows observed were:

-- first login

1.
DefaultIDTokenBuilder::buildIDToken (org.wso2.carbon.identity.openidconnect)
DefaultIDTokenBuilder::getSubjectClaim (org.wso2.carbon.identity.openidconnect)
OIDCClaimUtil::getSubjectClaimCachedAgainstAccessToken (org.wso2.carbon.identity.openidconnect)
OIDCClaimUtil::getSubjectClaimCachedAgainstAccessToken (org.wso2.carbon.identity.openidconnect)
AuthorizationGrantCache::replaceFromTokenId (org.wso2.carbon.identity.oauth.cache)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
** subject is retrieved from the AuthenticatedUser object

2.
DefaultIDTokenBuilder::buildIDToken (org.wso2.carbon.identity.openidconnect)
DefaultIDTokenBuilder::handleOIDCCustomClaims (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::handleCustomClaims (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::getUserClaimsInOIDCDialect (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::getCachedUserAttributes (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::getUserAttributesCachedAgainstToken (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::getUserAttributesCachedAgainstToken (org.wso2.carbon.identity.openidconnect)
AuthorizationGrantCache::getValueFromCacheByToken (org.wso2.carbon.identity.oauth.cache)
AuthorizationGrantCache::replaceFromTokenId (org.wso2.carbon.identity.oauth.cache)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
** user atributes are retrieved from the authorization code cache

-- refresh token:

** cache entry with the old token's token id (cache key) is invalidated (cleared), 
but its data (cache entry value) is added back to cache against the new access token value [4]. 
entire access token value is added as the cache key in the local cache but persisted in the session data store using the token id [5]

1.
DefaultIDTokenBuilder::buildIDToken (org.wso2.carbon.identity.openidconnect)
DefaultIDTokenBuilder::getSubjectClaim (org.wso2.carbon.identity.openidconnect)
OIDCClaimUtil::getSubjectClaimCachedAgainstAccessToken (org.wso2.carbon.identity.openidconnect)
OIDCClaimUtil::getSubjectClaimCachedAgainstAccessToken (org.wso2.carbon.identity.openidconnect)
AuthorizationGrantCache::replaceFromTokenId (org.wso2.carbon.identity.oauth.cache)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
** subject is retrieved from the AuthenticatedUser object

2.
DefaultIDTokenBuilder::buildIDToken (org.wso2.carbon.identity.openidconnect)
DefaultIDTokenBuilder::getAuthorizationGrantCacheEntryFromToken (org.wso2.carbon.identity.openidconnect)
AuthorizationGrantCache::getValueFromCacheByToken (org.wso2.carbon.identity.oauth.cache)
AuthorizationGrantCache::replaceFromTokenId (org.wso2.carbon.identity.oauth.cache)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
** returns null

3.
DefaultIDTokenBuilder::buildIDToken (org.wso2.carbon.identity.openidconnect)
DefaultIDTokenBuilder::handleOIDCCustomClaims (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::handleCustomClaims (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::getUserClaimsInOIDCDialect (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::getUserClaimsInOIDCDialect (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::getUserAttributesCachedAgainstToken (org.wso2.carbon.identity.openidconnect)
DefaultOIDCClaimsCallbackHandler::getUserAttributesCachedAgainstToken (org.wso2.carbon.identity.openidconnect)
AuthorizationGrantCache::getValueFromCacheByToken (org.wso2.carbon.identity.oauth.cache)
AuthorizationGrantCache::replaceFromTokenId (org.wso2.carbon.identity.oauth.cache)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
AccessTokenDAOImpl::getTokenIdByAccessToken (org.wso2.carbon.identity.oauth2.dao)
** returns null

4. 
** there was an attempt to retrieve from the previous access token [6], but the cache entry was already invalidated at the beginning of the operation.

How to reproduce:

Setup two Identity Server instances, one as an SP (we'll call this instance '9444') and the other as an IdP (we'll call this instance '9443') for federated authentication. I used SAML for federated authentication, but it should also work with OIDC (as long as the 'openid' scope is provided)
In 9444, check 'Supported by default' for some claims, create a user, and populate these claims. Additionally, add those claims in the SP's requested claims to provide the 9443 instance with some claims during federated authentication.
In 9443, add the configuration below to disable all caches and ensure all data is always retrieved from the database:
```
[server]
default_cache_timeout = "1m"
default_realm_cache_timeout = "1m"
force_local_cache = false
```

[authorization_manager.properties] AuthorizationCacheEnabled = "false"

[governance_data] enable_cache = false

[cache] framework_session_context_cache.enable = false authentication_context_cache.enable = false authentication_request_cache.enable = false authentication_result_cache.enable = false authentication_error_cache.enable = false app_info_cache.enable = false authorization_grant_cache.enable = false jwks_cache.enable = false oauth_cache.enable = false oauth_scope_cache.enable = false oauth_session_data_cache.enable = false saml_sso_participant_cache.enable = false saml_sso_session_index_cache.enable = false saml_sso_session_data_cache.enable = false service_provider_cache.enable = false service_provider_cache_id.enable = false service_provider_cache_inbound_auth.enable = false provisioning_connector_cache.enable = false provisioning_entity_cache.enable = false service_provider_provisioning_connector_cache.enable = false idp_cache_by_auth_property.enable = false idp_cache_by_hri.enable = false idp_cache_by_name.enable = false

4. In 9443, create an OIDC service provider (used OAuth2 Playground for this), and select the 9444 instance as the Federated Authenticator under the Local & Outbound Authentication Configuration tab.
5. Log in into the OIDC application ('openid' scope is mandatory for getting an ID Token), and retrieve its refresh token value.
6. Use the cURL command below to get the refresh token response (note the scopes must match the first login's):
```sh
curl -k "https://localhost:9443/oauth2/token?scope=openid" \ 
    -d "grant_type=refresh_token&refresh_token=<REFRESH_TOKEN>" \
    -u <CLIENT_ID>:<CLIENT_SECRET> \
    -H "Content-Type: application/x-www-form-urlencoded"

Parse the ID Token's JWT value and observe the missing claims.

Expected behavior: The ID token should have the user attributes populated accordingly.

Environment information:

Product Version: Tested on 5.11.0 but faulty logic still exists in master branch

[1] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/v6.4.111/components/org.wso2.carbon.identity.oauth/src/main/java/org/wso2/carbon/identity/openidconnect/DefaultOIDCClaimsCallbackHandler.java#L632 [2] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/v6.4.111/components/org.wso2.carbon.identity.oauth/src/main/java/org/wso2/carbon/identity/oauth/cache/AuthorizationGrantCache.java#L207 [3] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/master/components/org.wso2.carbon.identity.oauth/src/main/java/org/wso2/carbon/identity/oauth2/dao/AccessTokenDAOImpl.java#L2464 [4] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/v6.4.111/components/org.wso2.carbon.identity.oauth/src/main/java/org/wso2/carbon/identity/oauth2/token/handlers/grant/RefreshGrantHandler.java#L634-L636 [5] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/master/components/org.wso2.carbon.identity.oauth/src/main/java/org/wso2/carbon/identity/oauth/cache/AuthorizationGrantCache.java#L84 [6] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/master/components/org.wso2.carbon.identity.oauth/src/main/java/org/wso2/carbon/identity/openidconnect/DefaultOIDCClaimsCallbackHandler.java#L350-L363

wso2 / product-is

Missing user attributes in refresh token response's ID token if Application's Token Issuer is JWT #20517

Product Version: Tested on 5.11.0 but faulty logic still exists in master branch