spring-projects / spring-boot

Spring Boot helps you to create Spring-powered, production-grade applications and services with absolute minimum fuss.
https://spring.io/projects/spring-boot
Apache License 2.0
75.32k stars 40.72k forks source link

SimpleClientHttpRequest works different from previous SimpleBufferingClientHttpRequest. Cause of that now get: java.io.IOException: insufficient data written #39037

Closed ghost closed 10 months ago

ghost commented 10 months ago

Since spring boot 3.2 there are changes for RestTemplate to use new SimpleClientHttpRequest SimpleClientHttpRequest.executeInternal is different from before SimpleBufferingClientHttpRequest.executeInternal main problem is in logic

this.connection.setFixedLengthStreamingMode

In SimpleClientHttpRequest to setFixedLengthStreamingMode we take from headers content-lenght In SimpleBufferingClientHttpRequest we used predefined buffer 1024.

Cause of this change always get: java.io.IOException: insufficient data written on post.

Is this bug ? Or expected change ?

bclozel commented 10 months ago

This is a known change, see this section of the upgrade guide:

To reduce memory usage in RestClient and RestTemplate, most ClientHttpRequestFactory implementations no longer buffer request bodies before sending them to the server. As a result, for certain content types such as JSON, the contents size is no longer known, and a Content-Length header is no longer set. If you would like to buffer request bodies like before, simply wrap the ClientHttpRequestFactory you are using in a BufferingClientHttpRequestFactory.

ghost commented 10 months ago

Hi, @bclozel

Thanks for fast response.

Before writing issue. Checked upgrade guide info and found your described solution it doesn't solve problem at least in my side. It was already tested as guide describe.

new RestTemplateBuilder().requestFactory(() -> new BufferingClientHttpRequestFactory(new SimpleClientHttpRequestFactory()))

Problem still persists: java.io.IOException: insufficient data written. content-lenght: is incorrectly taken and calculated.

Only solution what I found was to calculate and preset correct content-lenght in headers before calling exchange. Additional i think it's same problem: https://github.com/spring-cloud/spring-cloud-gateway/issues/3154

bclozel commented 10 months ago

If your problem is similar to https://github.com/spring-cloud/spring-cloud-gateway/issues/3154, it means you are using RestTemplate as a client for proxying requests and that you are copying the Content-Length header back and that this turns out to be the wrong length. If anything, this means that you should not set the Content-Length header yourself unless you are sure about the actual body length.

ghost commented 10 months ago

Yes it's so. I'm using it as proxy.

public ResponseEntity<?> post(final String postfix, final Map<String, ?> body, final HttpServletRequest req) { final HttpHeaders headers = getHeaders(req); final ResponseEntity<?> response = restTemplate.exchange("url", HttpMethod.POST, new HttpEntity<Map<String, ?>>(body, headers), Object.class); return getResult(response); }

At moment i'm forced to do content-lenght calculations in getHeaders. To fix problem which are introduced since spring boot 3.2.
bclozel commented 10 months ago

I think you should move this to StackOverflow and get some advice there. Your implementation is probably problematic in many other ways (security headers or other sensitive headers, see https://github.com/spring-projects/spring-framework/issues/21523). This is not a Spring bug and you should review your implementation.