search

wso2 / product-is

Welcome to the WSO2 Identity Server source code! For info on working with the WSO2 Identity Server repository and contributing code, click the link below.
http://wso2.github.io/
Apache License 2.0
742 stars 720 forks source link

DCR application creation fails with MTLS #19858

Open RivinduM opened 6 months ago

RivinduM commented 6 months ago

Describe the issue: When sending the transport certificates with DCR POST call, it fails with a 500 internal server error with the below error log.

[2024-02-27 13:24:22,768] [07a51ad3-d052-453e-9af1-6165841790b3] ERROR {org.wso2.carbon.identity.oauth2.dcr.endpoint.impl.RegisterApiServiceImpl} - Internal Server Error org.wso2.carbon.identity.oauth.dcr.exception.DCRMException: Error while creating service provider: FAPI DCR app2 in tenant: carbon.super
    at org.wso2.carbon.identity.oauth.dcr.service.DCRMService.createServiceProvider(DCRMService.java:826)
    at org.wso2.carbon.identity.oauth.dcr.service.DCRMService.createServiceProvider(DCRMService.java:732)
    at org.wso2.carbon.identity.oauth.dcr.service.DCRMService.createOAuthApplication(DCRMService.java:479)
    at org.wso2.carbon.identity.oauth.dcr.service.DCRMService.registerApplication(DCRMService.java:158)
    at org.wso2.carbon.identity.oauth2.dcr.endpoint.impl.RegisterApiServiceImpl.registerApplication(RegisterApiServiceImpl.java:93)
    at org.wso2.carbon.identity.oauth2.dcr.endpoint.RegisterApi.registerApplication(RegisterApi.java:79)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265)
    at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
    at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:225)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:304)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:217)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:555)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:279)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
    at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:119)
    at org.wso2.carbon.identity.context.rewrite.valve.OrganizationContextRewriteValve.invoke(OrganizationContextRewriteValve.java:123)
    at org.wso2.carbon.tomcat.ext.valves.SameSiteCookieValve.invoke(SameSiteCookieValve.java:38)
    at org.wso2.carbon.identity.cors.valve.CORSValve.invoke(CORSValve.java:83)
    at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:211)
    at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:144)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:110)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:71)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:152)
    at org.wso2.carbon.extension.identity.x509Certificate.valve.X509CertificateAuthenticationValve.invoke(X509CertificateAuthenticationValve.java:59)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:670)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:63)
    at org.wso2.carbon.tomcat.ext.valves.RequestEncodingValve.invoke(RequestEncodingValve.java:49)
    at org.wso2.carbon.tomcat.ext.valves.RequestCorrelationIdValve.invoke(RequestCorrelationIdValve.java:137)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1794)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.wso2.carbon.identity.application.common.IdentityApplicationManagementException: Error resolving service provider owner.
    at org.wso2.carbon.identity.application.mgt.ApplicationManagementServiceImpl.lambda$createApplicationWithTemplate$0(ApplicationManagementServiceImpl.java:237)
    at java.base/java.util.Optional.orElseThrow(Optional.java:408)
    at org.wso2.carbon.identity.application.mgt.ApplicationManagementServiceImpl.createApplicationWithTemplate(ApplicationManagementServiceImpl.java:236)
    at org.wso2.carbon.identity.oauth.dcr.service.DCRMService.createServiceProvider(DCRMService.java:821)
    ... 62 more

[2024-02-27 13:24:22,787] [07a51ad3-d052-453e-9af1-6165841790b3]  WARN {org.apache.cxf.jaxrs.impl.WebApplicationExceptionMapper} - org.wso2.carbon.identity.oauth2.dcr.endpoint.exceptions.DCRMEndpointException: HTTP 500 Internal Server Error
    at org.wso2.carbon.identity.oauth2.dcr.endpoint.util.DCRMUtils.buildDCRMEndpointException(DCRMUtils.java:276)
    at org.wso2.carbon.identity.oauth2.dcr.endpoint.util.DCRMUtils.handleErrorResponse(DCRMUtils.java:196)
    at org.wso2.carbon.identity.oauth2.dcr.endpoint.impl.RegisterApiServiceImpl.registerApplication(RegisterApiServiceImpl.java:104)
    at org.wso2.carbon.identity.oauth2.dcr.endpoint.RegisterApi.registerApplication(RegisterApi.java:79)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265)
    at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
    at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:225)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:304)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:217)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:555)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:279)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
    at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:119)
    at org.wso2.carbon.identity.context.rewrite.valve.OrganizationContextRewriteValve.invoke(OrganizationContextRewriteValve.java:123)
    at org.wso2.carbon.tomcat.ext.valves.SameSiteCookieValve.invoke(SameSiteCookieValve.java:38)
    at org.wso2.carbon.identity.cors.valve.CORSValve.invoke(CORSValve.java:83)
    at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:211)
    at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:144)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:110)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:71)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:152)
    at org.wso2.carbon.extension.identity.x509Certificate.valve.X509CertificateAuthenticationValve.invoke(X509CertificateAuthenticationValve.java:59)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:670)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:63)
    at org.wso2.carbon.tomcat.ext.valves.RequestEncodingValve.invoke(RequestEncodingValve.java:49)
    at org.wso2.carbon.tomcat.ext.valves.RequestCorrelationIdValve.invoke(RequestCorrelationIdValve.java:137)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1794)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:829)

The reason for the issue is, we are setting the application owner by retrieving the username from the ThreadLocalCarbonContext[1], which is null when the MTLS authenticator is engaged.

Screenshot 2024-02-27 at 12 40 27

Since the user is not present when the MTLS authenticator is engaged, [2] fails to set the username to the context.

When the basic auth is used, this works fine.

Screenshot 2024-02-27 at 12 41 36

[1] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/master/components/org.wso2.carbon.identity.oauth.dcr/src/main/java/org/wso2/carbon/identity/oauth/dcr/service/DCRMService.java#L446 [2] https://github.com/wso2-extensions/identity-carbon-auth-rest/blob/master/components/org.wso2.carbon.identity.auth.service/src/main/java/org/wso2/carbon/identity/auth/service/handler/AuthenticationHandler.java#L108

How to reproduce:

  1. Try out the DCR post call with MTLS certificates. 
    
curl --location 'https://localhost:9443/api/identity/oauth2/dcr/v1.1/register' \
--cert <TRANSPORT_PUBLIC_KEY_FILE_PATH> --key <TRANSPORT_PRIVATE_KEY_FILE_PATH> \

--header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data '{ "redirect_uris": [ "https://abc/redirect1", "https://abc/redirect2" ], "client_name": "FAPI DCR app2", "grant_types": [ "client_credentials", "authorization_code" ] }'


2.  500 internal server error will be responded with the above error log

**Expected behavior:**
<!-- A clear and concise description of what you expected to happen. -->

**Environment information** (_Please complete the following information; remove  any unnecessary fields_) **:**
 - Product Version: IS 7.0.0-RC6
 - OS: [e.g., Windows, Linux, Mac]
 - Database: [e.g., MySQL, H2]
 - Userstore: [e.g., LDAP, JDBC]

---

### Optional Fields

**Related issues:**
<!-- Any related issues from this/other repositories-->

**Suggested labels:**
<!-- Only to be used by non-members -->
NiTR0-CH4RG3R commented 1 month ago

Hello,

I'm currently developing a sample Open Banking Toolkit based on the UK Specification on top of APIM 4.3.0 and IS 7.0.0. I have a situation where I need to call the Identity Server DCR endpoint from the APIM manager to create a Service Provider as part of the Open Banking DCR flow. I am attaching the Basic Auth header to the request before it reaches the Identity Server. Despite including this header, the Identity Server runs into this same issue.

The problem seems to be related to the APIM sending its certificate to the Identity Server when making the request. When I send the request through Postman without specifying any certificates, it works as expected.

I see there are no updates posted regarding this issue since it was opened. Since, this is impacting our development, could you please prioritize this issue and provide any updates or guidance?

NiTR0-CH4RG3R commented 1 month ago

Hello again,

I believe I have some new information that would be helpful to this. The problem seems to be in the authenticate() function in the AuthenticationManager class. When it calls the getFirstPriorityHandler() function in the HandlerManager class, it returns a ClientCertificateBasedAuthenticationHandler instance as the authentication handler when the client TLS certificate is set. This is because the getAuthenticationHandlers() returns a authentication handler array list with a ClientCertificateBasedAuthenticationHandler instance as the first element and getFirstPriorityHandler() returns the first handler in the list that returns true for the canHandle() function. (Please refer the following screenshot.)

with_certs_2_with_red_box

ClientCertificateBasedAuthenticationHandler class set the username returned from the WSO2-Identity-User http header.

client_certificate_handler_2_with_red_box

If we didn't send this header, the cert based auth handler will not set the user in the PrivilegedCarbonContext which then will cause the applicationOwner to be null in the DCRMService, which will cause the above issue.

I believe using ClientCertificateBasedAuthenticationHandler whenever the client certs are set, is the intended behavior for most use cases. But for DCR since we need an authenticated user, we shouldn't rely on a http header to retrieve the username. The DCR flow should have a way to figure out the authenticated user from the OAuth2 token or the basic Auth header as mentioned in the DCR documentation.

For my use case, I can temporary set the username in the WSO2-Identity-User header. I don't know if there is a way to configure the order of the authentication handlers from a config file (If there is please let me know). But sending the header is the only workaround I could find at the moment. Hope this get fixed soon.

The debug sessions shown in the screenshots are conducted on the https://github.com/wso2-extensions/identity-carbon-auth-rest/ repository and on the v1.9.4 tag.

P.S. You can reduce the priority of the ClientCertificateBasedAuthenticationHandler by changing the event.default_listener.client_certificate_authentication_handler.priority in the default.json file in the IS, or you can entirely disable the ClientCertificateBasedAuthenticationHandler by setting event.default_listener.client_certificate_authentication_handler.enable to false. (You can also override the default setting from the deployment.toml as well)

sadilchamishka commented 1 month ago

Hi @NiTR0-CH4RG3R

We can configure the certificate based authentication handler to use the "CN" of the certificate as the authenticated user [1].

The below sample toml config can be used if required.

[intermediate_cert_validation]
enable=true
cert_cns=['wso2isintcert', 'localhost']

[1] - https://wso2docs.atlassian.net/wiki/spaces/IS580/pages/41128359/Authenticating+and+Authorizing+REST+APIs

Screenshot 2024-08-02 at 06 57 27