gateway api cannot be accessed via curl when content length >1024 [100-continue]

[celesky]

python/java/php[cur]--------->springcloud gateway------[service discovery]----->myservice

springboot verison 2.1.1.RELEASE if the content length >1024, curl post request will stuck. I had used python,java ,postman to access my api ,they worked well but when i use php to access the api ,it will stucked the request, because my php program used curl to make http request.

i write the simple demo code in this repo https://github.com/celesky/problem-demo.git branch:curlbug

note: the demo code in branch : curlbug

1. first start eureka ,it will run at http://localhost:1111

2.second start sms-service ,it will run at http://localhost:8080 you can access the api with post method: http://localhost:8080/sms/send or exe this shell for test https://github.com/celesky/problem-demo/blob/curlbug/shell/curl_directly_tomcat.sh

3.third start gateway-server,it will run at http://localhost:7001

now exe this shell for test by curl ,it will stuck https://github.com/celesky/problem-demo/blob/curlbug/shell/curl_via_gateway.sh

or exe this python script ,it will return success quikly https://github.com/celesky/problem-demo/blob/curlbug/shell/python_via_gateway.py

-------look at this-------latest updated----------- on my mac os, i get this working now I just add -H 'Expect:' to the http header ,then it never get stuck again. Generally this is used to ignore http-100-contionue when httpserver does not support http-100-contionue ,why i must set it here, it makes me confused......

curl -X POST \
  http://localhost:7001/sms-service/sms/send \
  -H 'cache-control: no-cache' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
  -H 'Expect:' \
  -F content=211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131211231312312312313121123131231231231312112313123123123131

[ryanjbaxter]

We have no requirement that the expect header be present. We have used curl successfully a number of times to make requests to the gateway. Sounds like it’s something in your environment.

[celesky]

ok…, will the gateway support the http 100-continue in the future？ you can reproduce it by set the post form body param's length more than 1024 via curl. the request will be stuck

[ryanjbaxter]

I'm confused now is the bug with curl or a bug with the Expect header?

[celesky]

in my opinion, it's not curl's bug ,100-continue mechanism is a part of http protocol, curl makes it effective in default . and we can ignore it by add Expect header actually , tomcat or some other webserver can support it in default configuration

here are some ralated links: https://httpstatusdogs.com/100-continue https://stackoverflow.com/questions/463144/php-http-post-fails-when-curl-data-1024

Finally, I want to say maybe it's not a important problem ,but It wastes me a lot of time to check it and find it out 😆

[ryanjbaxter]

As I expected this is a problem with cURL. I added some logging to your sms-service and used cURL to make the request, here is what the headers look like on that request

2019-01-15 14:30:22.925 DEBUG 8642 --- [nio-8080-exec-3] o.s.w.f.CommonsRequestLoggingFilter      : Before request [uri=/sms/send;headers={accept=[*/*], content-type=[multipart/form-data; charset=utf-8; boundary=__X_PAW_BOUNDARY__; boundary=------------------------f88c9d8b73d6fdc7], expect=[100-continue], user-agent=[curl/7.54.0], content-length=[1510], type=[premium], forwarded=[proto=http;host="localhost:7001";for="0:0:0:0:0:0:0:1:63267"], x-forwarded-for=[0:0:0:0:0:0:0:1], x-forwarded-proto=[http], x-forwarded-prefix=[/sms-service], x-forwarded-port=[7001], x-forwarded-host=[localhost:7001], host=[192.168.1.138:8080]}]

Notice when using curl the expect=[100-continue] header is present.

When using other clients (in this case PAW) the expect header is not present

2019-01-15 14:32:44.777 DEBUG 8642 --- [nio-8080-exec-6] o.s.w.f.CommonsRequestLoggingFilter      : Before request [uri=/sms/send;headers={user-agent=[Paw/3.1.8 (Macintosh; OS X/10.14.2) GCDHTTPRequest], content-length=[1466], content-type=[multipart/form-data; charset=utf-8; boundary=__X_PAW_BOUNDARY__], forwarded=[proto=http;host="localhost:7001";for="127.0.0.1:49580"], x-forwarded-for=[127.0.0.1], x-forwarded-proto=[http], x-forwarded-prefix=[/sms-service], x-forwarded-port=[7001], x-forwarded-host=[localhost:7001], host=[192.168.1.138:8080]}]

This is why setting the expect header when using cURL makes it work.

[andrewfinnell]

This is presenting us an issue as well, and is not just a cURL problem. The .NET HttpClient also sends this header by default. And when the header is sent, the communications between a Service behind the Spring Cloud Gateway completely break. It not only breaks when the content is > 1024, but when the header is present at all.

This can be easily demonstrated by deploying the Spring Cloud Gateway, Route it to a WebFlux service, then call the the gateway endpoint with a .NET HttpClient.

If i must, I can create sample applications to demonstrate this, but the Spring Cloud Gateway absolutely does not work when the Expect 100 Continue header is sent, and it should.

[ryanjbaxter]

@andrewfinnell could you recreate this just using a regular Spring Boot WebFlux app or is it only when Spring Cloud Gateway is on the classpath?

[michaldo]

Hi @ryanjbaxter I read all issues related to header Expect 100-continue and this one seems be most active.

Answering your question, regardless how problem is caused, by curl, by .NET client or whatever, the flow is:

1. Client call cloud gateway with header Expect 100-continue
2. Gateway responds HTTP 100 continue
3. Client complete the call with request body to gateway
4. Gateway forward request to the target server, for example Spring Boot over Tomcat
5. Target server responds HTTP 100 continue
6. Gateway forward response HTTP 100 continue to the client. The client skip the response, as the request was already continued. Finally, client hung up

How the flow should look like? The simplest answer is

1. Client calls cloud gateway with header Expect 100-continue
2. Gateway responds HTTP 417 expectation failed - i am primitive gateway, expect 100 is too complicated
3. Client, according to rfc7231, retries the request without header Expect 100-continue. Further processing completes normally.

The more complicated flow could look like:

1. Client calls cloud gateway with header Expect 100-continue
2. Gateway, according to rfc2616,

If a proxy receives a request that includes an Expect request- header field with the "100-continue" expectation, and the proxy either knows that the next-hop server complies with HTTP/1.1 or higher, or does not know the HTTP version of the next-hop server, it MUST forward the request, including the Expect header field.

forwards request to target server

3. Target server responds HTTP 100 continue
4. Gateway forwards response to client
5. Client completes request with body
6. Gateway forwards body to target server,
7. Target server responds the response. Gateway forwards response to client

It is worth to highlight that there is known workaround: remove header expect when request forwarded to target server.

Feel free to ask for more details or demonstration case

[ryanjbaxter]

@michaldo thanks for this. Sounds like we need to open an issue to change the behavior of the gateway, would you mind doing that?

[andrewfinnell]

@ryanjbaxter my apologies for the delay. I had not seen the notification for your message but I did see this latest one. I’ll try to be more vigilant.

It does sound like it the behavior needs to change. The backend service seems to handle the Expect just fine, it’s only when the Spring Gateway is in the middle. If an issue needs to be opened and the other gentleman isn’t doing it let me know and I’ll open it.

[michaldo]

Created: https://github.com/spring-cloud/spring-cloud-gateway/issues/1337