search

spring-projects / spring-hateoas

Spring HATEOAS - Library to support implementing representations for hyper-text driven REST web services.
https://spring.io/projects/spring-hateoas
Apache License 2.0
1.03k stars 475 forks source link

JSON decoding issues after upgrade to Spring Cloud 2023.0.2 #2158

Closed kschlesselmann closed 1 month ago

kschlesselmann commented 3 months ago

After upgrading to Spring Cloud 2023.0.2 a lot of tests in our services fail with

`JSON decoding error: Trailing token (of type FIELD_NAME) found after value (bound as `org.springframework.hateoas.Link`): not allowed as per `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`

if they try to read the response body to a class that inherits from RepresentationModel.

As far as I can tell the HypermediaWebTestClientConfigurer is somehow involved. If I reomve it from our WebTestClient the response body can be read and the test fails due to class mismatches.

Everything here is on the reactive stack. Do you have any suggestions what's going on here?

pcornelissen commented 3 months ago

Using WebMVC the same error occured. I reverted back to 2023.0.1 until I know hot to fix that. And I'm using the generic wrappers and not the inheritance

odrotbohm commented 2 months ago

Can you provide a minimal reproducer that shows the error in a test case?

pcornelissen commented 2 months ago

As it prevents me from upgrading, ~~I started to dig deeper and it gets weirder. When the service runs in intelliJ I can't trigger the exception. It just happen s when the service is run via mvn outside of the IDE and sadly when it runs in the cluster...~~ (I can reproduce it in intelliJ now as well)

pcornelissen commented 2 months ago

OK, more digging. In my case it was triggered by a request in a webMVC application using webclient (as this service is older than restclient...). This I had no helpful context in the exception. I rewrote the part to use restclient and I could find that this is a request to another service, where I map a part of the response to an entity.

The response is:

{"id":"bfd1d723-0656-4b34-96a4-ce736c018eb3","version":14,"name":"Hydraulik Schneider Demo","logoUrl":"fd70beb2-c64e-4551-8607-afb7adaf1a3e","logoAssetId":null,"homepageUrl":"http://hydraulik-schneider.de","publicContentDomain":"http://hydraulikschneider.public.localhost","billingAddress":{"name":"Hydraulik Schneider AG","additionalNameRow":null,"street":"Beispielstraße","houseIdentifier":"42z","zipCode":"54321","city":"Beispielhausen","state":null,"country":"Deutschland"},"presets":{"colors":{"primary":"#00589C"},"general":{},"blockDefaults":null,"activeCss":null,"draftCss":null,"cssClassNames":null,"imprint":null,"dataProtectionOfficer":null},"auditMetaData":{"createdBy":"demo","createdDate":"2024-05-28T18:15:21.255939Z","modifiedBy":"tenant_all_roles@orchit-dev.de","modificationDate":"2024-07-03T11:02:09.989323Z"},"_links":{"self":{"href":"http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3"},"fetchCustomCss":{"href":"http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3/customcss{?draft}","templated":true},"fetchCustomCssNames":{"href":"http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3/customcss/names"},"fetchViaContentDomain":{"href":"http://localhost:9087/?contentDomain=http%3A%2F%2Fhydraulikschneider.public.localhost"},"updateTenant":{"href":"http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3","type":"PUT"}}}

This is not readable, so here is it formatted:

{
  "id": "bfd1d723-0656-4b34-96a4-ce736c018eb3",
  "version": 14,
  "name": "Hydraulik Schneider Demo",
  "logoUrl": "fd70beb2-c64e-4551-8607-afb7adaf1a3e",
  "logoAssetId": null,
  "homepageUrl": "http://hydraulik-schneider.de",
  "publicContentDomain": "http://hydraulikschneider.public.localhost",
  "billingAddress": {
    "name": "Hydraulik Schneider AG",
    "additionalNameRow": null,
    "street": "Beispielstraße",
    "houseIdentifier": "42z",
    "zipCode": "54321",
    "city": "Beispielhausen",
    "state": null,
    "country": "Deutschland"
  },
  "presets": {
    "colors": {
      "primary": "#00589C"
    },
    "general": {},
    "blockDefaults": null,
    "activeCss": null,
    "draftCss": null,
    "cssClassNames": null,
    "imprint": null,
    "dataProtectionOfficer": null
  },
  "auditMetaData": {
    "createdBy": "demo",
    "createdDate": "2024-05-28T18:15:21.255939Z",
    "modifiedBy": "tenant_all_roles@orchit-dev.de",
    "modificationDate": "2024-07-03T11:02:09.989323Z"
  },
  "_links": {
    "self": {
      "href": "http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3"
    },
    "fetchCustomCss": {
      "href": "http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3/customcss{?draft}",
      "templated": true
    },
    "fetchCustomCssNames": {
      "href": "http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3/customcss/names"
    },
    "fetchViaContentDomain": {
      "href": "http://localhost:9087/?contentDomain=http%3A%2F%2Fhydraulikschneider.public.localhost"
    },
    "updateTenant": {
      "href": "http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3",
      "type": "PUT"
    }
  }
}

I can't see any problem here. And both RestClient and Webclient fail with the same exception. Funny enough, when I let spring provide a ObjectMapper I can map to the class without problem. So there seems to be something fishy with the objectmapper that is used by default in webclient and restclient.

I am not providing my own objectmapper, as far as I can see to the context. And the problem is not present in 2023.0.1.

The error is triggered in the Jackson2HalModule while doing result.add(jp.readValueAs(Link.class).withRel(relation)); (Line 607) while reading the fetchCustomCss link. I have no clue what may trigger that.

kschlesselmann commented 2 months ago

@pcornelissen Is there something I could (easily) test on our side to provide more information? Sadly I had no time to dig deeper but it's blocking our update as well :-(

pcornelissen commented 2 months ago

Well, I have exhausted the time I can invest for now. I have reverted to 2023.0.1 and the problem is gone.

As a workaround, you can try to fetch the result as string and use the objectmapper yourself, but I ran into the problem that the objectmapper was not configured for hateoas (although I used the builder) and this lead to the fact that the links of the entity were not parsed. But that may be a viable workaround with a proper object mapper.

It seems the problem is purely on the clientside. i have compared the input that came over the wire and it looked the same no matter if the sending service was on 2023.0.2 or .1

If you find a solution, please report here. I get anxious when I am too far behind with updates ;-)

odrotbohm commented 2 months ago

I'm afraid that without a stack trace, some information about which Spring Cloud components are involved or anything that allows us to reproduce the issue, it's difficult to even remotely diagnose what the problem could be. Can anyone provide that information?

pcornelissen commented 2 months ago

Well, a stacktrace is easy:

2024-07-09T11:57:17.430+02:00 ERROR 34758 --- [Tenant] [mcat-handler-78] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.web.client.RestClientException: Error while extracting response for type [org.springframework.hateoas.EntityModel<MYPACKAGES_ANONYMIZED.BlanketContract>] and content type [application/hal+json]] with root cause

com.fasterxml.jackson.databind.exc.MismatchedInputException: Trailing token (of type FIELD_NAME) found after value (bound as `org.springframework.hateoas.Link`): not allowed as per `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 743] (through reference chain: org.springframework.hateoas.EntityModel["_links"])
    at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.DeserializationContext.reportTrailingTokens(DeserializationContext.java:1840) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.ObjectMapper._verifyNoTrailingTokens(ObjectMapper.java:5013) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4886) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3035) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.core.JsonParser.readValueAs(JsonParser.java:2453) ~[jackson-core-2.17.1.jar:2.17.1]
    at org.springframework.hateoas.mediatype.hal.Jackson2HalModule$HalLinkListDeserializer.deserialize(Jackson2HalModule.java:607) ~[spring-hateoas-2.3.0.jar:2.3.0]
    at org.springframework.hateoas.mediatype.hal.Jackson2HalModule$HalLinkListDeserializer.deserialize(Jackson2HalModule.java:554) ~[spring-hateoas-2.3.0.jar:2.3.0]
    at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeWithUnwrapped(BeanDeserializer.java:740) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:343) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2125) ~[jackson-databind-2.17.1.jar:2.17.1]
    at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1501) ~[jackson-databind-2.17.1.jar:2.17.1]
    at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:395) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:354) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:213) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.readBody(DefaultRestClient.java:688) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.body(DefaultRestClient.java:642) ~[spring-web-6.1.8.jar:6.1.8]
    at MYPACKAGES_ANONYMIZED.ContractsServiceRestRepository.fetchBlanketContract(ContractsServiceRestRepository.java:72) ~[classes/:na]
    at MYPACKAGES_ANONYMIZED.PublicInfoRestController.fetchPublicInfo(PublicInfoRestController.java:32) ~[classes/:na]
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:926) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:831) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) ~[tomcat-embed-core-10.1.24.jar:6.0]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.24.jar:6.0]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.springframework.web.filter.ForwardedHeaderFilter.doFilterInternal(ForwardedHeaderFilter.java:173) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$FilterObservation$SimpleFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:479) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:340) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator.lambda$wrapSecured$0(ObservationFilterChainDecorator.java:82) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:128) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:179) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.oauth2.server.resource.web.authentication.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:145) ~[spring-security-oauth2-resource-server-6.3.0.jar:6.3.0]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:82) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:69) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$0(ObservationFilterChainDecorator.java:323) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:224) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:233) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:191) ~[spring-security-web-6.3.0.jar:6.3.0]
    at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.servlet.handler.HandlerMappingIntrospector.lambda$createCacheFilter$3(HandlerMappingIntrospector.java:195) ~[spring-webmvc-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.CompositeFilter.doFilter(CompositeFilter.java:74) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.security.config.annotation.web.configuration.WebMvcSecurityConfiguration$CompositeFilterChainProxy.doFilter(WebMvcSecurityConfiguration.java:230) ~[spring-security-config-6.3.0.jar:6.3.0]
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:352) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:268) ~[spring-web-6.1.8.jar:6.1.8]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.1.8.jar:6.1.8]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:389) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
    at java.base/java.lang.VirtualThread.run(VirtualThread.java:309) ~[na:na]

The only thing I have to change to trigger this is switch from 2023.0.1 to 2023.0.2 in the service that triggers this error. No other change is necessary, the "source service" can remain on 2023.0.1 or be at 2023.0.2 as well, it doesn't matter.

odrotbohm commented 2 months ago

What's weird is that you seem to be able to pinpoint the issue on the Spring Cloud update, but there's no Spring Cloud involved in the stacktrace. Can you elaborate what you mean with "source service"? On which side does the update cause an issue? Are you saying the system serving the response, alters the response as an effect of the upgrade so that the client breaks? Or is it the client that starts failing to read the response?

dbernsau commented 2 months ago

Same issue here after the switch to 2023.0.2. We run integrations tests using TestRestTemplate.excahange(...).

Parsing the response into proper Java object fails now: 

org.springframework.web.client.RestClientException: Error while extracting response for type [class com.unboundid.scim2.common.messages.ErrorResponse] and content type [application/json]
        at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:119)
        at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:1159)
        at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:1142)
        at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:892)
        at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:790)
        at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:672)
        at org.springframework.boot.test.web.client.TestRestTemplate.exchange(TestRestTemplate.java:711)
        ...
Caused by: org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Trailing token (of type FIELD_NAME) found after value (bound as `java.lang.Integer`): not allowed as per `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`
        at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:406)
        at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:354)
        at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:104)
        ... 10 more
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Trailing token (of type FIELD_NAME) found after value (bound as `java.lang.Integer`): not allowed as per `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`
 at [Source: (org.springframework.util.StreamUtils$NonClosingInputStream); line: 1, column: 103] (through reference chain: com.unboundid.scim2.common.messages.ErrorResponse["status"])
        at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
        at com.fasterxml.jackson.databind.DeserializationContext.reportTrailingTokens(DeserializationContext.java:1825)
        at com.fasterxml.jackson.databind.ObjectMapper._verifyNoTrailingTokens(ObjectMapper.java:4933)
        at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4806)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2974)
        at com.fasterxml.jackson.core.JsonParser.readValueAs(JsonParser.java:2363)
        at com.unboundid.scim2.common.utils.StatusDeserializer.deserialize(StatusDeserializer.java:41)
        at com.unboundid.scim2.common.utils.StatusDeserializer.deserialize(StatusDeserializer.java:31)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1419)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
        at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2105)
        at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1481)
        at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:395)
        ... 12 more

I debugged into the whole thing and the key is within ObjectMapper._readValue(...) there is the part 

        if (cfg.isEnabled(DeserializationFeature.FAIL_ON_TRAILING_TOKENS)) {
            this._verifyNoTrailingTokens(p, ctxt, valueType);
        }

and with the old version the config did not have FAIL_ON_TRAILING_TOKENS enabled. And since the JSON we parse has trailing tokens (which is OK and matches the target object structure) the whole thing fails now.

pcornelissen commented 2 months ago

Ah ok. Wenn I meant that this is communication between two microservices. The client is the one that throws the exception and the server spits out the same result, no matter which spring cloud version is in use. As far as I have seen it, the response from the server is identical with both versions.

The client on the other hand fails when reading the second link in the result with the latest spring cloud. (I stepped through and this is what it looked like. The bytestreams make it a bit hard to follow the exact position in the document)

odrotbohm commented 2 months ago

Is there any chance you capture some of the payload and provide a minimal reproducer trying to statically read that? From what you describe the (reduced) content put into a file and an interaction with an ObjectMapper configured by a full app bootstrap should be able to reproduce the problem, shouldn't it?

pcornelissen commented 2 months ago

The payload from my project is in the comments before

dbernsau commented 2 months ago

Here is my payload: 

{"schemas":["urn:ietf:params:scim:api:messages:2.0:Error"],"id":null,"externalId":null,"status":"404","meta":null,"scimType":null,"detail":"User with uuid 03031ece-d670-4f23-adc8-48247aa078f2 doesn't exist"}

which will be deserialized into com.unboundid.scim2.common.messages.ErrorResponse which defines a specific deserializer (String => Integer) for the status field. And exactly that goes wrong.

 @JsonSerialize(
        using = StatusSerializer.class
    )
    @JsonDeserialize(
        using = StatusDeserializer.class
    )
    private final int status;
dbernsau commented 1 month ago

Spring Cloud 2023.0.3 fixed the issue for me

Alien2150 commented 1 month ago

Spring Cloud 2023.0.3 fixed the issue for me

Confirmed. It was fixed by spring cloud: https://github.com/spring-cloud/spring-cloud-function/issues/1148

pcornelissen commented 1 month ago

Confirmed from me as well, the problem is gone

odrotbohm commented 1 month ago

Looks like this can be closed then. Thanks everyone for their input!