spring-projects / spring-framework

Spring Framework
https://spring.io/projects/spring-framework
Apache License 2.0
56.47k stars 38.09k forks source link

Spring RestClient returns extra empty header with value :status:200 #32297

Closed alibutt30 closed 8 months ago

alibutt30 commented 8 months ago

Affects: spring v6.1.3 and possibly v6.1.4 as well


Hello,

We are using the RestClient that was introduced in spring boot v3.2. When we call https://www.google.com we receive an extra empty header with value :status:200

We did a comparison with the old RestTemplate and the response headers from RestTemplate do not have this empty header or a status header altogether.

The code is as below: (Also attaching the project as well)

@GetMapping("/test")
    ResponseEntity<String> test() {
        RestTemplate restTemplate = new RestTemplate();
        String fooResourceUrl = "https://www.google.com";
        ResponseEntity<String> restTemplateResponse = restTemplate.getForEntity(fooResourceUrl, String.class);
        HttpHeaders restTemplateResponseHeaders = restTemplateResponse.getHeaders();
        System.out.println("\n \n ======= Rest Template Headers =======\n \n");
        System.out.println(restTemplateResponseHeaders);

        // RestClient has :status header returned
        RestClient restClient = RestClient.create();
        ResponseEntity<String> restClientResponse = restClient
                .get()
                .uri("https://www.google.com")
                .retrieve()
                .toEntity(String.class);
        HttpHeaders restClientResponseHeaders = restClientResponse.getHeaders();
        System.out.println("\n \n ======= Rest Client Headers =======\n \n");
        System.out.println(restClientResponseHeaders);
        return restClientResponse;
    }

The sample project is attached here: restClientTesting.zip

Below is the response headers from both clients printed for comparison. (see the first header of RestClient with :status:200)

Response

======= Rest Template Headers =======

[Date:"Tue, 20 Feb 2024 08:32:32 GMT", Expires:"-1", Cache-Control:"private, max-age=0", Content-Type:"text/html; charset=ISO-8859-1", Content-Security-Policy-Report-Only:"object-src 'none';base-uri 'self';script-src 'nonce-pIPc4PNueaFN4dmg2XfPDA' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp", P3P:"CP="This is not a P3P policy! See g.co/p3phelp for more info."", Server:"gws", X-XSS-Protection:"0", X-Frame-Options:"SAMEORIGIN", Set-Cookie:"SOCS=CAAaBgiA18-uBg; expires=Fri, 21-Mar-2025 08:32:32 GMT; path=/; domain=.google.com; Secure; SameSite=lax", "AEC=Ae3NU9OyNq3cE0mX2hAmGqapJ7o2AqaTWfaSIHI9Lj_H4kXU08NsTWHxFuI; expires=Sun, 18-Aug-2024 08:32:32 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax", "__Secure-ENID=17.SE=lHSRawN99fBHUEDf2taMi4UMXvQJvRIJNEZGIWBi0Dz5-LBZ8DKwdIhmgyML-FEz1DDZ5ea1Sn0dhfg1Ob6MTlzGIE7sjXMs2HJ7eETLr65MLHRAxTNpyqTJmyupa0tRcivlBNWBsZUX2WfNFw49ZIVtZnDimCUpowKj76daeoo; expires=Sat, 22-Mar-2025 00:50:50 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax", Alt-Svc:"h3=":443"; ma=2592000,h3-29=":443"; ma=2592000", Accept-Ranges:"none", Vary:"Accept-Encoding", Transfer-Encoding:"chunked"]

 ======= Rest Client Headers =======

[:status:"200", accept-ranges:"none", alt-svc:"h3=":443"; ma=2592000,h3-29=":443"; ma=2592000", cache-control:"private, max-age=0", content-security-policy-report-only:"object-src 'none';base-uri 'self';script-src 'nonce-949vlaT_7uDM0hnp-rAMnA' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp", content-type:"text/html; charset=ISO-8859-1", date:"Tue, 20 Feb 2024 08:32:32 GMT", expires:"-1", p3p:"CP="This is not a P3P policy! See g.co/p3phelp for more info."", server:"gws", set-cookie:"SOCS=CAAaBgiA18-uBg; expires=Fri, 21-Mar-2025 08:32:32 GMT; path=/; domain=.google.com; Secure; SameSite=lax", "AEC=Ae3NU9NZdnSuoqchV5daP_ZMChDzJzT5etUfdw8gbvfBOnTmvK3JAVF4-gA; expires=Sun, 18-Aug-2024 08:32:32 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax", "__Secure-ENID=17.SE=nZWI_IRdH71Y6TXEysDqjAQBvag53hAjALuM1NRqVghOWnjWv4buyhTbfEIcOHn-10ubwDqFVScuqC1mC3Fh0js__Sv6ymjnxoup8IA6KCakekJKQl-85vbqOp4_StBPdPZDO_LTDQ9qGEnp4l4L5bxPrS58AlQoyrL6z51tans; expires=Sat, 22-Mar-2025 00:50:50 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax", vary:"Accept-Encoding", x-frame-options:"SAMEORIGIN", x-xss-protection:"0"]
snicoll commented 8 months ago

We did a comparison with the old RestTemplate and the response headers from RestTemplate do not have this empty header or a status header altogether.

RestClient uses RestTemplate behind the scenes. The sample you've shared is not an apple-to-apple comparison. If you create the RestClient as follows, the headers are the same:

RestClient restClient = RestClient.create(restTemplate);

If you are migrating from a RestTemplate custom setup and you want to use the RestClient API, it's better to create the RestClient that way.

That being said, you've described a difference, but not how it is problematic.

alibutt30 commented 8 months ago

Hello @snicoll

Thank you for your quick response and explaining the difference between RestClient and RestTemplate

The problem is the empty header with value status:200.

Therefore i wanted to know if this is working as expected ? or this is a bug?

image

Ragamuffin85 commented 8 months ago

Thanks for raising this topic - I was stuggling with it as well.

In my case, I wanted to replace and unify the RestTemplate with the new RestClient and therefore removed the RestTemplate completly - Unfortunately out of nowhere the new header appeared and gave me some headaches.

So for me it would also be intersting if this is the expected behavior and if so, if we can get rid of the new status header since our clients fail during whilst parsinf this header field

@snicoll Thanks for the fast response and the awesome support - You guys doing a great job!

bclozel commented 8 months ago

As explained by @snicoll, the difference here is about how the client is created. By default, RestClient will use JDK's java.net.http.HttpClient if it is available and no other library is present. On the other hand, RestTemplate will use java.net.HttpURLConnection.

I guess the difference you're seeing here is about the client libraries themselves, possibly one supported HTTP/2 and the other not?

snicoll commented 8 months ago

RestClient in your sample uses the JDK client. However, that screenshot above looks like the tool you are using is not parsing things properly. On our side that header has a key of :status with a value of 200. Perhaps the tool is confused by the colon at the beginning?

snicoll commented 8 months ago

I am afraid this has nothing to do with Spring. I've updated your sample to use the JDK client directly:

HttpClient jdkClient = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder(URI.create("https://www.google.com")).build();
HttpResponse<Void> response = jdkClient.send(request, BodyHandlers.discarding());
java.net.http.HttpHeaders headers = response.headers();
System.out.println("\n \n ======= JDK Client Headers =======\n \n");
System.out.println(headers);

Which produces:

======= JDK Client Headers =======

java.net.http.HttpHeaders@b3730b4c { {:status=[200], accept-ranges=[none], alt-svc=[h3=":443"; ma=2592000,h3-29=":443"; ma=2592000], cache-control=[private, max-age=0], content-security-policy-report-only=[object-src 'none';base-uri 'self';script-src 'nonce-CRVSORCd7_Uqhz2sjStRhQ' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp], content-type=[text/html; charset=ISO-8859-1], date=[Tue, 20 Feb 2024 10:10:59 GMT], expires=[-1], p3p=[CP="This is not a P3P policy! See g.co/p3phelp for more info."], server=[gws], set-cookie=[SOCS=CAAaBgiA18-uBg; expires=Fri, 21-Mar-2025 10:10:59 GMT; path=/; domain=.google.com; Secure; SameSite=lax, AEC=Ae3NU9PAJ5xGcE3sCxDs5s9Yiu9Z92ANyXoisxSSjRcD2uBqI3Wmp_rSgg; expires=Sun, 18-Aug-2024 10:10:59 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax, __Secure-ENID=17.SE=oQuDzMBYlDJrpwYMn5DcYf5PUrkIDMSspQLJb_OBwbtrQJawOpMATBYiBg5MP4e8ZBVXLvh7O_-ZCHFeTfK3iMFSFMekc0OhhqaBM7sCDMXa9qUET0Cgc3TNRZkiMZH1Ck6dGJKZxpjRtOoxgmCpsZ2CctPm4MMuiraAAHFQmhcGErc5wg; expires=Sat, 22-Mar-2025 02:29:17 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax], vary=[Accept-Encoding], x-frame-options=[SAMEORIGIN], x-xss-protection=[0]} }

I also confirm that the header is not empty as you claim. It has a key of :status. I am going to close this issue and would encourage you to file a report against the JDK. I haven't found anything that points to an expected behavior.

Ragamuffin85 commented 8 months ago

@snicoll

I followed your advise and created a OpenJDK Bug Ticket https://bugs.java.com/bugdatabase/view_bug?bug_id=JDK-8326418

With the outcome, that both components behave as expected but respect different HTTP standards.

HttpClient supports HTTP/2 (from JDK11 onwards) responds with a valid pseudo header (':status') whereas HttpURLConnection only supports HTTP/1.1 which is not intended to support this kind of header.

So long story short - Both responses are valid but we need to take care of the applied standard.