TryGetDestination error

fearmadara commented 2 weeks ago

Issue Description

Important information:

Your code

public Destination getDestination(String destinationName) {
    Try<Destination> destinationTry = tryGetDestination(destinationName);
    if (destinationTry.isSuccess()) {
        try {
            return destinationTry.get();
        } catch (DestinationAccessException e) {
            throw e.getDestinationName() != null ? e
                    : new DestinationAccessException(destinationName, e.getMessage(), e.getCause());
        }
    } else {
        throw new DestinationAccessException(destinationName, "Cannot retrieve Destination details",
                destinationTry.getCause());
    }
}

Expected outcome Destination object
Actual outcome Cannot retrieve Destination details
Steps attempted to resolve the issue
In case of issues with any of our VDMs:
- What happens when a request is performed directly via an HTTP client tool such as Postman? tried running the service locally, we get the same error
- In case that succeeds please provide details on the request such as URL, query parameters, header parameters, prerequisite requests (e.g. CSRF token), etc.
Potentially missing information (open questions you might have)

Impact / Priority

Our E2E test are failing, and we notice that the issue is coming in our dev space and not in prod space

Affected development phase: Development

Impact: Impaired

Timeline: e.g. Go-Live is in 12 weeks.

Error Message

com.sap.cloud.sdk.cloudplatform.resilience.ResilienceRuntimeException: com.sap.cloud.sdk.cloudplatform.thread.exception.ThreadContextExecutionException: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationAccessException: Could not resolve destination to Destination Service on behalf of TECHNICAL_USER_CURRENT_TENANT.

Project Details

SDK Version:
com.sap.cloud.sdk.cloudplatform.connectivity 5.10.0
Link to GitHub repo: https://github.tools.sap/CALMRun/x-oa-service/blob/main/srv/src/main/java/com/sap/crun/oa/global/util/DestinationServiceImpl.java
Project type, for example:
- [X] CAP Project
- [ ] SDK Maven Archetype
- [ ] None of the above:
Platform:
- [ ] Cloud Foundry
- [X] Deploy with Confidence (Cloud Foundry)
- [ ] None of the above:

Checklist

[ ] Checked out the documentation and Stack Overflow
[ ] Description provided with all relevant information
[ ] Exception and stack trace provided
[ ] Attached debug logs
[ ] Attached dependency tree
[ ] Provided Cloud SDK version & link to relevant source code

KavithaSiva commented 2 weeks ago

Hi @fearmadara,

Could you please add the complete stack trace to see the root cause of the exception?

fearmadara commented 1 week ago

HI @KavithaSiva ,

We see that now the issue is not appearing, hence not able to get the trace, also we have noticed that this issue occurs randomly and then disappears.

Will provide the logs when it occurs again.

Thank You Regards Akash

fearmadara commented 2 days ago

DestinationNotFoundException: Destination could not be found for path destination-configuration/v1/destinations/OPS_OA_AUTOPI.

2024-07-08 11:39:13,211 INFO [Timer-0] - [] [com.sap.calm.metering.accumulation.startegies.TimerStrategy] : Timer is up! flushing with tenant id [] 2024-07-08 11:39:13,213 INFO [Timer-0] - [] [com.sap.calm.metering.accumulation.startegies.TimerStrategy] : The tenant id is inital 2024-07-08 11:39:14,943 ERROR [http-nio-8080-exec-2] - [tenant_id=06ab7953-5f4b-40c7-90ab-4325b2991ebb, request=/odata/v4/definition/getAutoPiCommandsList(destination='OPS_OA_AUTOPI'), component_id=8ac35ce7-ed5a-48da-a636-d74d095fd061, component_name=x-oa-srv-01903092-d429-7614-7ed1-b0c750c2bac0, organization_name=calm-1provider-dev-eu10-004, sap_passport=2A54482A0300E60000782D6F612D7372762D30313930333039322D643432392D373631342D376564310000616E6F6E796D6F757300000000000000000000000000000000000000000000005341505F434C4F55445F53444B5F5245515545535400000000000000000000000000000000000000000B782D6F612D7372762D30313930333039322D643432392D373631342D3765643132356362346639376266373834306538393462363931393565646465373434362020200002B87B04463B4F499AA76925E1DA13B02DFB7FD8A3E6174FAA8E5575C176E9F33D00000000000000002A54482A, component_type=application, space_name=calm-dev-eu10-004, organization_id=9466f430-e999-4f97-81ff-52f2b5af9362, component_instance=0, correlation_id=befb46ee-5220-4a22-a6cc-482af897853b, space_id=da58109a-b0c0-4730-9b23-f64de97e44d5, container_id=-, tenant_subdomain=calm-dev-eu10-004-relcfeatures-e2e-test-basic] [com.sap.crun.oa.definition.services.ExternalAutoPiDefinitionService] : AutoPi Error:: Can't access destination : OPS_OA_AUTOPI com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationAccessException: Cannot retrieve Destination details at com.sap.crun.oa.global.util.DestinationServiceImpl.getDestination(DestinationServiceImpl.java:39) at com.sap.crun.oa.global.util.ExternalConnectivity.getDestination(ExternalConnectivity.java:37) at com.sap.crun.oa.global.util.ExternalConnectivity.geAutoPiDestination(ExternalConnectivity.java:42) at com.sap.crun.oa.definition.services.ExternalAutoPiDefinitionService.getHttpDestination(ExternalAutoPiDefinitionService.java:110) at com.sap.crun.oa.definition.services.ExternalAutoPiDefinitionService.getCommandsList(ExternalAutoPiDefinitionService.java:66) at com.sap.crun.oa.definition.handlers.DefinitionHandler.onGetAutoPiCommandsList(DefinitionHandler.java:737) at com.sap.cds.services.impl.handlerregistry.HandlerRegistryTools$DescribedHandler.process(HandlerRegistryTools.java:165) at com.sap.cds.services.impl.ServiceImpl.lambda$createOnHandlerChain$4(ServiceImpl.java:269) at com.sap.cds.services.impl.ServiceImpl.dispatch(ServiceImpl.java:236) at com.sap.cds.services.impl.ServiceImpl.emit(ServiceImpl.java:177) at com.sap.cds.services.ServiceDelegator.emit(ServiceDelegator.java:33) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.sap.cds.services.impl.cds.TypedCqnServiceInvocationHandler.invoke(TypedCqnServiceInvocationHandler.java:70) at jdk.proxy2/jdk.proxy2.$Proxy85.emit(Unknown Source) at com.sap.cds.adapter.odata.v4.processors.CdsProcessor.operation(CdsProcessor.java:674) at com.sap.cds.adapter.odata.v4.processors.CdsProcessor.delegateRequest(CdsProcessor.java:194) at com.sap.cds.adapter.odata.v4.processors.CdsProcessor.lambda$processRequest$1(CdsProcessor.java:153) at com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.lambda$run$3(RequestContextRunnerImpl.java:213) at com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:272) at com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:212) at com.sap.cds.adapter.odata.v4.processors.CdsProcessor.processRequest(CdsProcessor.java:150) at com.sap.cds.adapter.odata.v4.processors.CdsProcessor.lambda$processRequest$0(CdsProcessor.java:141) at com.sap.cds.services.impl.runtime.ChangeSetContextRunnerImpl.open(ChangeSetContextRunnerImpl.java:62) at com.sap.cds.services.impl.runtime.ChangeSetContextRunnerImpl.run(ChangeSetContextRunnerImpl.java:41) at com.sap.cds.adapter.odata.v4.processors.CdsProcessor.processRequest(CdsProcessor.java:140) at com.sap.cds.adapter.odata.v4.processors.AbstractODataProcessor.processRequest(AbstractODataProcessor.java:163) at com.sap.cds.adapter.odata.v4.processors.FastODataProcessor.processEntities(FastODataProcessor.java:161) at com.sap.cds.adapter.odata.v4.processors.OlingoProcessor.readEntityCollection(OlingoProcessor.java:117) at org.apache.olingo.server.core.ODataDispatcher.handleEntityCollectionDispatching(ODataDispatcher.java:542) at org.apache.olingo.server.core.ODataDispatcher.handleEntityDispatching(ODataDispatcher.java:524) at org.apache.olingo.server.core.ODataDispatcher.handleFunctionDispatching(ODataDispatcher.java:200) at org.apache.olingo.server.core.ODataDispatcher.handleResourceDispatching(ODataDispatcher.java:149) at org.apache.olingo.server.core.ODataDispatcher.dispatch(ODataDispatcher.java:120) at org.apache.olingo.server.core.ODataHandlerImpl.processInternal(ODataHandlerImpl.java:170) at org.apache.olingo.server.core.ODataHandlerImpl.process(ODataHandlerImpl.java:86) at org.apache.olingo.server.core.ODataHttpHandlerImpl.process(ODataHttpHandlerImpl.java:75) at org.apache.olingo.server.core.ODataHttpHandlerImpl.process(ODataHttpHandlerImpl.java:89) at com.sap.cds.adapter.odata.v4.CdsODataV4Servlet.lambda$service$0(CdsODataV4Servlet.java:130) at com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.lambda$run$3(RequestContextRunnerImpl.java:213) at com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:272) at com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:212) at com.sap.cds.adapter.odata.v4.CdsODataV4Servlet.service(CdsODataV4Servlet.java:89) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) at com.sap.cds.framework.spring.config.adapter.ServletAdapterRestController.handleRequest(ServletAdapterRestController.java:30) at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:51) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at com.sap.crun.jobscheduler.context.JobSchedulerContextPersistenceFilter.doFilterInternal(JobSchedulerContextPersistenceFilter.java:67) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at com.sap.calm.x.foundation.passport.PassportRequestFilter.doFilterInternal(PassportRequestFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at com.sap.calm.x.foundation.sanitizer.RestJSONSanitizer.doFilterInternal(RestJSONSanitizer.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at com.sap.calm.x.foundation.sanitizer.RestJSONSanitizer.doFilterInternal(RestJSONSanitizer.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at com.sap.dwc.util.headers.http.DwcHeaderRequestFilter.doFilterInternal(DwcHeaderRequestFilter.java:43) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108) at org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231) at org.springframework.security.web.ObservationFilterChainDecorator$FilterObservation$SimpleFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:479) at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:340) at org.springframework.security.web.ObservationFilterChainDecorator.lambda$wrapSecured$0(ObservationFilterChainDecorator.java:82) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:128) at org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:179) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:82) at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:69) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$0(ObservationFilterChainDecorator.java:323) at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:224) at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:233) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:191) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at org.springframework.web.servlet.handler.HandlerMappingIntrospector.lambda$createCacheFilter$3(HandlerMappingIntrospector.java:195) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at org.springframework.web.filter.CompositeFilter.doFilter(CompositeFilter.java:74) at org.springframework.security.config.annotation.web.configuration.WebMvcSecurityConfiguration$CompositeFilterChainProxy.doFilter(WebMvcSecurityConfiguration.java:230) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:352) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:268) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:107) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at com.sap.calm.x.foundation.fesr.FesrRequestFilter.doFilterInternal(FesrRequestFilter.java:18) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:731) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:389) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:904) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) at java.base/java.lang.Thread.run(Thread.java:840) Caused by: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException: Destination could not be found in any of the registered loaders. Check the suppressed exceptions and the logs above to see which loaders where queried. at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationLoaderChain.tryGetDestination(DestinationLoaderChain.java:110) at com.sap.crun.oa.global.util.DestinationServiceImpl.tryGetDestination(DestinationServiceImpl.java:46) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationLoader.tryGetDestination(DestinationLoader.java:37) at com.sap.crun.oa.global.util.DestinationServiceImpl.getDestination(DestinationServiceImpl.java:29) ... 186 common frames omitted Suppressed: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException: Could not find environment variable for name 'destinations'. at com.sap.cloud.sdk.cloudplatform.connectivity.EnvVarDestinationLoader.getJsonNodes(EnvVarDestinationLoader.java:138) at com.sap.cloud.sdk.cloudplatform.connectivity.EnvVarDestinationLoader.extractDestination(EnvVarDestinationLoader.java:162) at com.sap.cloud.sdk.cloudplatform.connectivity.EnvVarDestinationLoader.lambda$tryGetDestination$79e22d5b$1(EnvVarDestinationLoader.java:116) at io.vavr.control.Try.of(Try.java:75) at com.sap.cloud.sdk.cloudplatform.connectivity.EnvVarDestinationLoader.tryGetDestination(EnvVarDestinationLoader.java:116) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationLoaderChain.tryGetDestination(DestinationLoaderChain.java:89) ... 189 common frames omitted Suppressed: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException: com.sap.cloud.sdk.cloudplatform.resilience.ResilienceRuntimeException: com.sap.cloud.sdk.cloudplatform.thread.exception.ThreadContextExecutionException: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException: Destination could not be found for path destination-configuration/v1/destinations/OPS_OA_AUTOPI. at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.resilientCall(DestinationService.java:352) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.lambda$loadAndParseDestination$1(DestinationService.java:137) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationRetrievalStrategyResolver.lambda$prepareSupplier$4(DestinationRetrievalStrategyResolver.java:179) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationRetrieval.get(DestinationRetrieval.java:26) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.loadAndParseDestination(DestinationService.java:145) at com.sap.cloud.sdk.cloudplatform.connectivity.GetOrComputeSingleDestinationCommand.lambda$prepareCommand$0(GetOrComputeSingleDestinationCommand.java:70) at io.vavr.control.Try.of(Try.java:75) at io.vavr.control.Try.ofSupplier(Try.java:92) at com.sap.cloud.sdk.cloudplatform.connectivity.GetOrComputeSingleDestinationCommand.execute(GetOrComputeSingleDestinationCommand.java:157) at io.vavr.control.Try.flatMapTry(Try.java:490) at io.vavr.control.Try.flatMap(Try.java:472) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService$Cache.getOrComputeDestination(DestinationService.java:850) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.tryGetDestination(DestinationService.java:128) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationLoaderChain.tryGetDestination(DestinationLoaderChain.java:89) ... 189 common frames omitted Caused by: com.sap.cloud.sdk.cloudplatform.resilience.ResilienceRuntimeException: com.sap.cloud.sdk.cloudplatform.thread.exception.ThreadContextExecutionException: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException: Destination could not be found for path destination-configuration/v1/destinations/OPS_OA_AUTOPI. at com.sap.cloud.sdk.cloudplatform.resilience4j.Resilience4jDecorationStrategy.lambda$decorateCallable$3(Resilience4jDecorationStrategy.java:195) at io.vavr.control.Try.onFailure(Try.java:659) at com.sap.cloud.sdk.cloudplatform.resilience4j.Resilience4jDecorationStrategy.lambda$decorateCallable$4(Resilience4jDecorationStrategy.java:194) at io.vavr.control.Try.of(Try.java:75) at io.vavr.control.Try.ofCallable(Try.java:105) at com.sap.cloud.sdk.cloudplatform.resilience4j.Resilience4jDecorationStrategy.lambda$decorateSupplier$1(Resilience4jDecorationStrategy.java:160) at com.sap.cloud.sdk.cloudplatform.resilience.ResilienceDecorationStrategy.executeSupplier(ResilienceDecorationStrategy.java:103) at com.sap.cloud.sdk.cloudplatform.resilience.ResilienceDecorator.executeSupplier(ResilienceDecorator.java:201) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.resilientCall(DestinationService.java:348) ... 202 common frames omitted Caused by: com.sap.cloud.sdk.cloudplatform.thread.exception.ThreadContextExecutionException: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException: Destination could not be found for path destination-configuration/v1/destinations/OPS_OA_AUTOPI. at com.sap.cloud.sdk.cloudplatform.thread.ThreadContextExecutor.execute(ThreadContextExecutor.java:246) at com.sap.cloud.sdk.cloudplatform.thread.DefaultThreadContextExecutorService.lambda$decorate$0(DefaultThreadContextExecutorService.java:72) at com.sap.cds.integration.cloudsdk.decorator.CdsThreadContextDecorator.lambda$decorateCallable$0(CdsThreadContextDecorator.java:36) at com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:272) at com.sap.cds.integration.cloudsdk.decorator.CdsThreadContextDecorator.lambda$decorateCallable$1(CdsThreadContextDecorator.java:34) at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ... 1 common frames omitted Caused by: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException: Destination could not be found for path destination-configuration/v1/destinations/OPS_OA_AUTOPI. at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationServiceAdapter.handleResponse(DestinationServiceAdapter.java:156) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationServiceAdapter.getConfigurationAsJson(DestinationServiceAdapter.java:141) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.retrieveDestination(DestinationService.java:152) at com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.lambda$loadAndParseDestination$0(DestinationService.java:137) at com.sap.cloud.sdk.cloudplatform.resilience4j.Resilience4jDecorationStrategy.lambda$decorateCallable$2(Resilience4jDecorationStrategy.java:179) at io.github.resilience4j.bulkhead.Bulkhead.lambda$decorateCallable$5(Bulkhead.java:173) at com.sap.cloud.sdk.cloudplatform.thread.ThreadContextExecutor.call(ThreadContextExecutor.java:297) at com.sap.cloud.sdk.cloudplatform.thread.ThreadContextExecutor.execute(ThreadContextExecutor.java:240) ... 9 common frames omitted 2024-07-08 11:39:14,947 DEBUG [HikariPool-1 connection adder] - [] [com.zaxxer.hikari.pool.PoolBase] : HikariPool-1 - Failed to create/setup connection: SAP DBTech JDBC: Cannot connect to jdbc:sap://caedca73-0dd4-4879-b117-06d1a12a32ab.hna1.prod-eu10.hanacloud.ondemand.com:443 [Object is closed: com.sap.db.jdbc.SecureChannelSession@31cdd0d9 caedca73-0dd4-4879-b117-06d1a12a32ab-719ef-0.1dm.hna1.prod-eu10.hanacloud.ondemand.com:443 ConnectionID:0 SessionID:0 on sun.nio.ch.WindowsAsynchronousSocketChannelImpl[connected local=/[0:0:0:0:0:0:0:0]:51671 remote=caedca73-0dd4-4879-b117-06d1a12a32ab-719ef-0.1dm.hna1.prod-eu10.hanacloud.ondemand.com/3.123.104.204:443]: 10.53.210.70:51671 -> caedca73-0dd4-4879-b117-06d1a12a32ab-719ef-0.1dm.hna1.prod-eu10.hanacloud.ondemand.com:443 ConnectionID:0 SessionID:0]. 2024-07-08 11:39:14,947 DEBUG [HikariPool-1 connection adder] - [] [com.zaxxer.hikari.pool.HikariPool] : HikariPool-1 - Cannot acquire connection from data source com.sap.db.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: Cannot connect to jdbc:sap://caedca73-0dd4-4879-b117-06d1a12a32ab.hna1.prod-eu10.hanacloud.ondemand.com:443 [Object is closed: com.sap.db.jdbc.SecureChannelSession@31cdd0d9 caedca73-0dd4-4879-b117-06d1a12a32ab-719ef-0.1dm.hna1.prod-eu10.hanacloud.ondemand.com:443 ConnectionID:0 SessionID:0 on sun.nio.ch.WindowsAsynchronousSocketChannelImpl[connected local=/[0:0:0:0:0:0:0:0]:51671 remote=caedca73-0dd4-4879-b117-06d1a12a32ab-719ef-0.1dm.hna1.prod-eu10.hanacloud.ondemand.com/3.123.104.204:443]: 10.53.210.70:51671 -> caedca73-0dd4-4879-b117-06d1a12a32ab-719ef-0.1dm.hna1.prod-eu10.hanacloud.ondemand.com:443 ConnectionID:0 SessionID:0]. at com.sap.db.jdbc.exceptions.SQLExceptionSapDB._newInstance(SQLExceptionSapDB.java:126) at com.sap.db.jdbc.exceptions.SQLExceptionSapDB._newInstance(SQLExceptionSapDB.java:222) at com.sap.db.jdbc.exceptions.SQLExceptionSapDB.newInstance(SQLExceptionSapDB.java:50) at com.sap.db.jdbc.DriverSapDB._connect(DriverSapDB.java:3430) at com.sap.db.jdbc.DriverSapDB.connect(DriverSapDB.java:3013) at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:137) at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:360) at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:202) at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:461) at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:724) at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:703) at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) 2024-07-08 11:39:14,949 INFO [http-nio-8080-exec-2] - [tenant_id=06ab7953-5f4b-40c7-90ab-4325b2991ebb, request=/odata/v4/definition/getAutoPiCommandsList(destination='OPS_OA_AUTOPI'), component_id=8ac35ce7-ed5a-48da-a636-d74d095fd061, component_name=x-oa-srv-01903092-d429-7614-7ed1-b0c750c2bac0, organization_name=calm-1provider-dev-eu10-004, sap_passport=2A54482A0300E60000782D6F612D7372762D30313930333039322D643432392D373631342D376564310000616E6F6E796D6F757300000000000000000000000000000000000000000000005341505F434C4F55445F53444B5F5245515545535400000000000000000000000000000000000000000B782D6F612D7372762D30313930333039322D643432392D373631342D3765643132356362346639376266373834306538393462363931393565646465373434362020200002B87B04463B4F499AA76925E1DA13B02DFB7FD8A3E6174FAA8E5575C176E9F33D00000000000000002A54482A, component_type=application, space_name=calm-dev-eu10-004, organization_id=9466f430-e999-4f97-81ff-52f2b5af9362, component_instance=0, correlation_id=befb46ee-5220-4a22-a6cc-482af897853b, space_id=da58109a-b0c0-4730-9b23-f64de97e44d5, container_id=-, tenant_subdomain=calm-dev-eu10-004-relcfeatures-e2e-test-basic] [com.sap.cds.adapter.odata.v4.processors.CdsProcessor] : Exception marked the ChangeSet 3 as cancelled: Cannot access Automation Pilot destination "OPS_OA_AUTOPI". Make sure that the destination exists and that it is correctly configured in the destination service. (service 'DefinitionService', event 'getAutoPiCommandsList', entity '') 2024-07-08 11:39:17,480 DEBUG [HikariPool-1 connection adder] - [] [com.zaxxer.hikari.pool.HikariPool] : HikariPool-1 - Added connection com.sap.db.jdbc.HanaConnectionClean@22cda4b6[ID:257850] 2024-07-08 11:39:17,515 DEBUG [HikariPool-1 connection adder] - [] [com.zaxxer.hikari.pool.HikariPool] : HikariPool-1 - After adding stats (total=10, active=0, idle=10, waiting=0) 2024-07-08 11:39:17,515 DEBUG [HikariPool-1 connection adder] - [] [com.zaxxer.hikari.pool.HikariPool] : HikariPool-1 - Connection not added, stats (total=10, active=0, idle=10, waiting=0)

fearmadara commented 2 days ago

Hi @KavithaSiva ,

i have pasted the trace , could you please check on the issue

Thank You Regards Akash

KavithaSiva commented 2 days ago

Hi @fearmadara,

In the stack trace you included, I can see Destination could not be found for path destination-configuration/v1/destinations/OPS_OA_AUTOPI which indicates that the destination service responded with a 404 when queried for this particular destination. Is querying for the destination working when you use a REST client like for e.g. Postman?

Where have you created the destination OPS_OA_AUTOPI? Is it in the provider or any other subscriber sub-account? Try to print the output of AuthTokenAccessor.getCurrentToken().getJwt().getToken() just before you call your getDestination method. It could be that the destination is being queried in the wrong sub-account because of a different auth token.

Regards, Kavitha

fearmadara commented 1 day ago

"Caused by: com.sap.cloud.sdk.cloudplatform.resilience.ResilienceRuntimeException: java.util.concurrent.TimeoutException: TimeLimiter 'class com.sap.cloud.sdk.cloudplatform.connectivity.DestinationServicesingleDestResilience' recorded a timeout exception."

"com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationAccessException: Cannot retrieve Destination details", "\tat com.sap.crun.oa.global.util.DestinationServiceImpl.getDestination(DestinationServiceImpl.java:39)", "\tat com.sap.crun.oa.global.util.ExternalConnectivity.getDestination(ExternalConnectivity.java:37)", "\tat com.sap.crun.oa.global.util.ExternalConnectivity.geAutoPiDestination(ExternalConnectivity.java:42)", "\tat com.sap.crun.oa.definition.services.ExternalAutoPiDefinitionService.getHttpDestination(ExternalAutoPiDefinitionService.java:110)", "\tat com.sap.crun.oa.definition.services.ExternalAutoPiDefinitionService.getCommandsList(ExternalAutoPiDefinitionService.java:66)", "\tat com.sap.crun.oa.definition.handlers.DefinitionHandler.onGetAutoPiCommandsList(DefinitionHandler.java:737)", "\tat com.sap.cds.services.impl.handlerregistry.HandlerRegistryTools$DescribedHandler.process(HandlerRegistryTools.java:165)", "\tat com.sap.cds.services.impl.ServiceImpl.lambda$createOnHandlerChain$4(ServiceImpl.java:269)", "\tat com.sap.cds.services.impl.ServiceImpl.dispatch(ServiceImpl.java:236)", "\tat com.sap.cds.services.impl.ServiceImpl.emit(ServiceImpl.java:177)", "\tat com.sap.cds.services.ServiceDelegator.emit(ServiceDelegator.java:33)", "\tat jdk.internal.reflect.GeneratedMethodAccessor184.invoke(Unknown Source)", "\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "\tat java.base/java.lang.reflect.Method.invoke(Method.java:568)", "\tat com.sap.cds.services.impl.cds.TypedCqnServiceInvocationHandler.invoke(TypedCqnServiceInvocationHandler.java:70)", "\tat jdk.proxy2/jdk.proxy2.$Proxy84.emit(Unknown Source)", "\tat com.sap.cds.adapter.odata.v4.processors.CdsProcessor.operation(CdsProcessor.java:674)", "\tat com.sap.cds.adapter.odata.v4.processors.CdsProcessor.delegateRequest(CdsProcessor.java:194)", "\tat com.sap.cds.adapter.odata.v4.processors.CdsProcessor.lambda$processRequest$1(CdsProcessor.java:153)", "\tat com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.lambda$run$3(RequestContextRunnerImpl.java:213)", "\tat com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:272)", "\tat com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:212)", "\tat com.sap.cds.adapter.odata.v4.processors.CdsProcessor.processRequest(CdsProcessor.java:150)", "\tat com.sap.cds.adapter.odata.v4.processors.CdsProcessor.lambda$processRequest$0(CdsProcessor.java:141)", "\tat com.sap.cds.services.impl.runtime.ChangeSetContextRunnerImpl.open(ChangeSetContextRunnerImpl.java:62)", "\tat com.sap.cds.services.impl.runtime.ChangeSetContextRunnerImpl.run(ChangeSetContextRunnerImpl.java:41)", "\tat com.sap.cds.adapter.odata.v4.processors.CdsProcessor.processRequest(CdsProcessor.java:140)", "\tat com.sap.cds.adapter.odata.v4.processors.AbstractODataProcessor.processRequest(AbstractODataProcessor.java:163)", "\tat com.sap.cds.adapter.odata.v4.processors.FastODataProcessor.processEntities(FastODataProcessor.java:161)", "\tat com.sap.cds.adapter.odata.v4.processors.OlingoProcessor.readEntityCollection(OlingoProcessor.java:117)", "\tat org.apache.olingo.server.core.ODataDispatcher.handleEntityCollectionDispatching(ODataDispatcher.java:542)", "\tat org.apache.olingo.server.core.ODataDispatcher.handleEntityDispatching(ODataDispatcher.java:524)", "\tat org.apache.olingo.server.core.ODataDispatcher.handleFunctionDispatching(ODataDispatcher.java:200)", "\tat org.apache.olingo.server.core.ODataDispatcher.handleResourceDispatching(ODataDispatcher.java:149)", "\tat org.apache.olingo.server.core.ODataDispatcher.dispatch(ODataDispatcher.java:120)", "\tat org.apache.olingo.server.core.ODataHandlerImpl.processInternal(ODataHandlerImpl.java:170)", "\tat org.apache.olingo.server.core.ODataHandlerImpl.process(ODataHandlerImpl.java:86)", "\tat org.apache.olingo.server.core.ODataHttpHandlerImpl.process(ODataHttpHandlerImpl.java:75)", "\tat org.apache.olingo.server.core.ODataHttpHandlerImpl.process(ODataHttpHandlerImpl.java:89)", "\tat com.sap.cds.adapter.odata.v4.CdsODataV4Servlet.lambda$service$0(CdsODataV4Servlet.java:130)", "\tat com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.lambda$run$3(RequestContextRunnerImpl.java:213)", "\tat com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:272)", "\tat com.sap.cds.services.impl.runtime.RequestContextRunnerImpl.run(RequestContextRunnerImpl.java:212)", "\tat com.sap.cds.adapter.odata.v4.CdsODataV4Servlet.service(CdsODataV4Servlet.java:89)", "\tat jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)", "\tat com.sap.cds.framework.spring.config.adapter.ServletAdapterRestController.handleRequest(ServletAdapterRestController.java:30)", "\tat org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:51)", "\tat org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089)", "\tat org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979)", "\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)", "\tat org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903)", "\tat jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564)", "\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)", "\tat jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat com.sap.crun.jobscheduler.context.JobSchedulerContextPersistenceFilter.doFilterInternal(JobSchedulerContextPersistenceFilter.java:65)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat com.sap.calm.x.foundation.passport.PassportRequestFilter.doFilterInternal(PassportRequestFilter.java:55)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat com.sap.calm.x.foundation.sanitizer.RestJSONSanitizer.doFilterInternal(RestJSONSanitizer.java:96)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat com.sap.calm.x.foundation.sanitizer.RestJSONSanitizer.doFilterInternal(RestJSONSanitizer.java:96)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat com.sap.dwc.util.headers.http.DwcHeaderRequestFilter.doFilterInternal(DwcHeaderRequestFilter.java:43)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108)", "\tat org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$FilterObservation$SimpleFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:479)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:340)", "\tat org.springframework.security.web.ObservationFilterChainDecorator.lambda$wrapSecured$0(ObservationFilterChainDecorator.java:82)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:128)", "\tat org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)", "\tat org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:131)", "\tat org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:85)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:179)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat com.sap.dwc.util.mauth.tls.filter.SpringSecurityTlsMAuthRequestFilter.doFilterInternal(SpringSecurityTlsMAuthRequestFilter.java:56)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107)", "\tat org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)", "\tat org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:82)", "\tat org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:69)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$0(ObservationFilterChainDecorator.java:323)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:224)", "\tat org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)", "\tat org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:233)", "\tat org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:191)", "\tat org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113)", "\tat org.springframework.web.servlet.handler.HandlerMappingIntrospector.lambda$createCacheFilter$3(HandlerMappingIntrospector.java:195)", "\tat org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113)", "\tat org.springframework.web.filter.CompositeFilter.doFilter(CompositeFilter.java:74)", "\tat org.springframework.security.config.annotation.web.configuration.WebMvcSecurityConfiguration$CompositeFilterChainProxy.doFilter(WebMvcSecurityConfiguration.java:230)", "\tat org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:352)", "\tat org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:268)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat com.sap.calm.x.foundation.fesr.FesrRequestFilter.doFilterInternal(FesrRequestFilter.java:18)", "\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)", "\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)", "\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)", "\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)", "\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)", "\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)", "\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)", "\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)", "\tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)", "\tat org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:731)", "\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)", "\tat org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:389)", "\tat org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)", "\tat org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)", "\tat org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741)", "\tat org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)", "\tat org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)", "\tat org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)", "\tat org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)", "\tat java.base/java.lang.Thread.run(Thread.java:840)", "Caused by: com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationAccessException: Failed to get destination.", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.resilientCall(DestinationService.java:354)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.lambda$loadAndParseDestination$1(DestinationService.java:137)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationRetrievalStrategyResolver.lambda$prepareSupplier$5(DestinationRetrievalStrategyResolver.java:193)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationRetrieval.get(DestinationRetrieval.java:26)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.loadAndParseDestination(DestinationService.java:145)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.GetOrComputeSingleDestinationCommand.lambda$prepareCommand$0(GetOrComputeSingleDestinationCommand.java:70)", "\tat io.vavr.control.Try.of(Try.java:75)", "\tat io.vavr.control.Try.ofSupplier(Try.java:92)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.GetOrComputeSingleDestinationCommand.execute(GetOrComputeSingleDestinationCommand.java:157)", "\tat io.vavr.control.Try.flatMapTry(Try.java:490)", "\tat io.vavr.control.Try.flatMap(Try.java:472)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService$Cache.getOrComputeDestination(DestinationService.java:850)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.tryGetDestination(DestinationService.java:128)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationLoaderChain.tryGetDestination(DestinationLoaderChain.java:89)", "\tat com.sap.crun.oa.global.util.DestinationServiceImpl.tryGetDestination(DestinationServiceImpl.java:46)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationLoader.tryGetDestination(DestinationLoader.java:37)", "\tat com.sap.crun.oa.global.util.DestinationServiceImpl.getDestination(DestinationServiceImpl.java:29)", "\t... 198 more", "Caused by: com.sap.cloud.sdk.cloudplatform.resilience.ResilienceRuntimeException: java.util.concurrent.TimeoutException: TimeLimiter 'class com.sap.cloud.sdk.cloudplatform.connectivity.DestinationServicesingleDestResilience' recorded a timeout exception.", "\tat com.sap.cloud.sdk.cloudplatform.resilience4j.Resilience4jDecorationStrategy.lambda$decorateCallable$3(Resilience4jDecorationStrategy.java:195)", "\tat io.vavr.control.Try.onFailure(Try.java:659)", "\tat com.sap.cloud.sdk.cloudplatform.resilience4j.Resilience4jDecorationStrategy.lambda$decorateCallable$4(Resilience4jDecorationStrategy.java:194)", "\tat io.vavr.control.Try.of(Try.java:75)", "\tat io.vavr.control.Try.ofCallable(Try.java:105)", "\tat com.sap.cloud.sdk.cloudplatform.resilience4j.Resilience4jDecorationStrategy.lambda$decorateSupplier$1(Resilience4jDecorationStrategy.java:160)", "\tat com.sap.cloud.sdk.cloudplatform.resilience.ResilienceDecorationStrategy.executeSupplier(ResilienceDecorationStrategy.java:103)", "\tat com.sap.cloud.sdk.cloudplatform.resilience.ResilienceDecorator.executeSupplier(ResilienceDecorator.java:201)", "\tat com.sap.cloud.sdk.cloudplatform.connectivity.DestinationService.resilientCall(DestinationService.java:348)", "\t... 214 more", "Caused by: java.util.concurrent.TimeoutException: TimeLimiter 'class com.sap.cloud.sdk.cloudplatform.connectivity.DestinationServicesingleDestResilience' recorded a timeout exception.", "\tat java.base/java.util.concurrent.FutureTask.get(FutureTask.java:204)", "\tat io.github.resilience4j.timelimiter.internal.TimeLimiterImpl.lambda$decorateFutureSupplier$0(TimeLimiterImpl.java:46)", "\tat io.github.resilience4j.circuitbreaker.CircuitBreaker.lambda$decorateCallable$4(CircuitBreaker.java:168)", "\tat io.vavr.control.Try.of(Try.java:75)", "\tat io.vavr.control.Try.ofCallable(Try.java:105)", "\tat com.sap.cloud.sdk.cloudplatform.resilience4j.Resilience4jDecorationStrategy.lambda$decorateCallable$4(Resilience4jDecorationStrategy.java:190)", "\t... 220 more"

fearmadara commented 1 day ago

Hi @KavithaSiva ,

I have pasted the new log trace, we have added the destination, and it is in a subaccount.

KavithaSiva commented 1 day ago

Hi @fearmadara,

I can see a java.util.concurrent.TimeoutException: TimeLimiter 'class com.sap.cloud.sdk.cloudplatform.connectivity.DestinationServicesingleDestResilience' recorded a timeout exception." in your stack trace. This means that while querying the destination service the response took longer than 6 seconds(which is the default timeout value for destination retrievals).

I am not sure why the destination service takes so long to respond in your case. Can you maybe try out querying the destination service directly using a REST client like for e.g. Postman? And see if it takes long to respond?

As a side note you could also try tweaking the default timelimiter value for destination retrievals. Ensure that you use the linked code snippet once during application startup and all your subsequent calls to destination service should have an increased timeout value.

SAP / cloud-sdk-java