search

wso2 / product-apim

Welcome to the WSO2 API Manager source code! For info on working with the WSO2 API Manager repository and contributing code, click the link below.
http://wso2.github.io/
Apache License 2.0
845 stars 785 forks source link

Error while retrieving application - Cannot retrieve service provider for the given consumer key - #9290

Open BirahimFall opened 4 years ago

BirahimFall commented 4 years ago

TID: [-1234] [api/am/store] [2020-09-22 14:58:32,827] ERROR {org.wso2.carbon.apimgt.rest.api.store.v1.impl.ApplicationsApiServiceImpl} - Error while retrieving application 81db4f81-2304-4cb7-8ab8-540032e2dd7d org.wso2.carbon.apimgt.api.APIManagementException: Cannot retrieve service provider for the given consumer key : cZXqLHWjLaFO9HpwyuJg_IPMYfMa at org.wso2.carbon.apimgt.impl.AbstractKeyManager.handleException_aroundBody10(AbstractKeyManager.java:259) at org.wso2.carbon.apimgt.impl.AbstractKeyManager.handleException(AbstractKeyManager.java:257) at org.wso2.carbon.apimgt.impl.AMDefaultKeyManagerImpl.retrieveApplication_aroundBody10(AMDefaultKeyManagerImpl.java:291) at org.wso2.carbon.apimgt.impl.AMDefaultKeyManagerImpl.retrieveApplication(AMDefaultKeyManagerImpl.java:281) at org.wso2.carbon.apimgt.impl.AbstractAPIManager.getApplicationKeys_aroundBody214(AbstractAPIManager.java:3103) at org.wso2.carbon.apimgt.impl.AbstractAPIManager.getApplicationKeys(AbstractAPIManager.java:3092) at org.wso2.carbon.apimgt.impl.AbstractAPIManager.getApplicationByUUID_aroundBody126(AbstractAPIManager.java:1845) at org.wso2.carbon.apimgt.impl.AbstractAPIManager.getApplicationByUUID(AbstractAPIManager.java:1842) at org.wso2.carbon.apimgt.rest.api.store.v1.impl.ApplicationsApiServiceImpl.applicationsApplicationIdGet(ApplicationsApiServiceImpl.java:221) at org.wso2.carbon.apimgt.rest.api.store.v1.ApplicationsApi.applicationsApplicationIdGet(ApplicationsApi.java:145) at sun.reflect.GeneratedMethodAccessor290.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179) at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:193) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:103) at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59) at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96) at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267) at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234) at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208) at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160) at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:216) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:301) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:225) at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:276) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:86) at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:110) at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:75) at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99) at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49) at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62) at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:145) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57) at org.wso2.carbon.tomcat.ext.valves.RequestCorrelationIdValve.invoke(RequestCorrelationIdValve.java:119) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: org.wso2.carbon.apimgt.impl.kmclient.KeyManagerClientException: Received status code: 403 Reason: at org.wso2.carbon.apimgt.impl.kmclient.KMClientErrorDecoder.decode_aroundBody0(KMClientErrorDecoder.java:29) at org.wso2.carbon.apimgt.impl.kmclient.KMClientErrorDecoder.decode(KMClientErrorDecoder.java:27) at feign.AsyncResponseHandler.handleResponse(AsyncResponseHandler.java:96) at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:138) at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:89) at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:100) at com.sun.proxy.$Proxy480.getApplication(Unknown Source) at org.wso2.carbon.apimgt.impl.AMDefaultKeyManagerImpl.retrieveApplication_aroundBody10(AMDefaultKeyManagerImpl.java:288)

Step to reproduce the issue: Configure WSO2 APIM in SSO with Keycloak Connect to devportal in SSO Create an application browse application to see details, generate application consumer keys (sandbox) logout login again click to application button click to the already created application: cannot view application details and wsocarbon log, we can see : Error while retrieving application 81db4f81-2304-4cb7-8ab8-540032e2dd7d org.wso2.carbon.apimgt.api.APIManagementException: Cannot retrieve service provider for the given consumer key : cZXqLHWjLaFO9HpwyuJg_IPMYfMa

This bug is reproduced probalby after switching between local IDP and keycloak IDP

TheDigitalEagle commented 4 years ago

Did you find any work around? We are dealing with the same issue.

BirahimFall commented 4 years ago

I am not working on it right now but I suspect that the issue is related to H2 database. Le mardi 20 octobre 2020 à 06:38:41 UTC+2, Chad Kullhem notifications@github.com a écrit :

Did you find any work around? We are dealing with the same issue.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

TheDigitalEagle commented 4 years ago

We are using PostgreSQL. Seems that would rule out H2 being the cause. We are also using Keycloak in our setup so seems to be the only correlation between our issues.

BirahimFall commented 4 years ago

I am not 100% sure but I think the issue occurs after the following sequence:

We are using PostgreSQL. Seems that would rule out H2 being the cause. We are also using Keycloak in our setup so seems to be the only correlation between our issues.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

lkokila commented 3 years ago

This seems like a result of a missing configuration as reported in https://github.com/wso2/docs-apim/issues/2257.

[authentication.framework.extensions] provisioning_handler = "org.wso2.carbon.identity.application.authentication.framework.handler.provisioning.impl.SystemRolesRetainedProvisionHandler"

The above configuration is required in the deploment.toml file to sort this out by engaging the SystemRolesRetainedProvisionHandler.

anshulsood2006 commented 3 years ago

Any updates on this. I am also getting this error and not able to fix using above solution

[authentication.framework.extensions] provisioning_handler = "org.wso2.carbon.identity.application.authentication.framework.handler.provisioning.impl.SystemRolesRetainedProvisionHandler"

natechols commented 3 years ago

I have run into the same problem (in AM 3.2.0), also using Postgres, but in my case I'm attempting to pre-load the application into the database as raw SQL (an earlier version of which worked in AM 2.0.0). The provisioning_handler config fix had no effect for me either. FWIW I get the same error attempting to simply view the pre-loaded application in the devportal, but setting up an equally simple application from scratch does not have this problem.

The one difference between my problem and the OP is that they are seeing an underlying HTTP 401 error while I get HTTP 403:

TID: [-1234] [api/am/store] [2021-03-30 16:47:09,176] ERROR {org.wso2.carbon.apimgt.impl.AbstractKeyManager} - Cannot retrieve service provider for the given consumer key : *** org.wso2.carbon.apimgt.impl.kmclient.KeyManagerClientException: Received status code: 403 Reason:
        at org.wso2.carbon.apimgt.impl.kmclient.KMClientErrorDecoder.decode_aroundBody0(KMClientErrorDecoder.java:29)
        at org.wso2.carbon.apimgt.impl.kmclient.KMClientErrorDecoder.decode(KMClientErrorDecoder.java:27)
...

I have attempted to find the source of this error but the architecture of WSO2 makes it extremely difficult. With logging set to INFO level for log components there is no other mention of the 403 error, the HTTP request that led to it, or the failing key. I can't find anything in the DB queries either, and I've also tried comparing my DB setup to the devportal API calls but since those aren't documented it's difficult to tell what to look at.

anshulsood2006 commented 3 years ago

I have run into the same problem (in AM 3.2.0), also using Postgres, but in my case I'm attempting to pre-load the application into the database as raw SQL (an earlier version of which worked in AM 2.0.0). The provisioning_handler config fix had no effect for me either. FWIW I get the same error attempting to simply view the pre-loaded application in the devportal, but setting up an equally simple application from scratch does not have this problem.

The one difference between my problem and the OP is that they are seeing an underlying HTTP 401 error while I get HTTP 403:

TID: [-1234] [api/am/store] [2021-03-30 16:47:09,176] ERROR {org.wso2.carbon.apimgt.impl.AbstractKeyManager} - Cannot retrieve service provider for the given consumer key : *** org.wso2.carbon.apimgt.impl.kmclient.KeyManagerClientException: Received status code: 403 Reason:
        at org.wso2.carbon.apimgt.impl.kmclient.KMClientErrorDecoder.decode_aroundBody0(KMClientErrorDecoder.java:29)
        at org.wso2.carbon.apimgt.impl.kmclient.KMClientErrorDecoder.decode(KMClientErrorDecoder.java:27)
...

I have attempted to find the source of this error but the architecture of WSO2 makes it extremely difficult. With logging set to INFO level for log components there is no other mention of the 403 error, the HTTP request that led to it, or the failing key. I can't find anything in the DB queries either, and I've also tried comparing my DB setup to the devportal API calls but since those aren't documented it's difficult to tell what to look at.

Heylo there, I was actually able to resolve this issue for me by making manual entries for my registered application in the below tables (in shared_db )

I got hint by going through the documentation at path https://apim.docs.wso2.com/en/latest/administer/managing-users-and-roles/managing-user-roles/

It has been mentioned there as

**_"When a user creates an application and generates application keys, a role is created automatically in the following format.

"Application/__PRODUCTION" This is a special case of the internal role that is created for a particular service provider application. Only users who are assigned the application role permission can manage the corresponding service provider application.

These roles do not have any permissions assigned to it, but it is used to manage the visibility of the corresponding service provider that is created in the format of '_PRODUCTION' within the Key Manager. The created service provider is only visible to users with the latter mentioned role that has been generated automatically. Only if a user with admin privileges assigns the latter mentioned role to a user, will that user be able to view the details of the service provider that is created per application."**

natechols commented 3 years ago

Thank you @anshulsood2006! After experimenting some more I had figured out there was something in the shared and/or carbon dbs that I needed, but I hadn't narrowed it down to specific tables or records yet.

INSERT INTO um_hybrid_role VALUES (9, 'Application/admin_DefaultApplication_PRODUCTION', -1234);
INSERT INTO um_hybrid_user_role VALUES (7, 'admin', 9, -1234, 1);

Just out of curiosity, did you determine whether the provisioning_handler fix made any difference in your case? (I included it in my config but I haven't tried changing it back yet.)

anshulsood2006 commented 3 years ago

No @natechols , I infact tried provisioning_handler in the first stance and that literally didn't help me. So currently I don't have that entry in my deployment.toml.

BirahimFall commented 3 years ago

Adding the fix (below) in the deployment.toml has resolved our issue. [authentication.framework.extensions] provisioning_handler = "org.wso2.carbon.identity.application.authentication.framework.handler.provisioning.impl.SystemRolesRetainedProvisionHandler In our case, we remove all applications created prior to the issue.