ninenines / gun

HTTP/1.1, HTTP/2, Websocket client (and more) for Erlang/OTP.
ISC License
891 stars 232 forks source link

Unexpected HTTP 400 on possibly misconfigured CloudFront server #221

Closed 1player closed 4 years ago

1player commented 4 years ago

Note: this request has not been working with any Erlang/Elixir HTTP client, OK with curl and browsers.

The URL in question is https://partners.betvictor.mobi/en/sport/364/meetings and other paths under that hostname.

curl has no issues loading that URL, with either HTTP/1 or HTTP/2:

$ curl -I https://partners.betvictor.mobi/en/sport/364/meetings
HTTP/2 200
content-type: application/json; charset=utf-8
content-length: 33500
server: CloudFront
etag: "c2e2de84dbde206f90c83d400fcf67cc"
date: Tue, 10 Mar 2020 16:29:48 GMT
vary: Accept-Encoding
access-control-allow-origin: https://www.betvictor.com
access-control-allow-methods: GET, OPTIONS
access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept
x-cache: Miss from cloudfront
via: 1.1 e46d5e94093ff4a4a8b6b4e0d2227693.cloudfront.net (CloudFront)
x-amz-cf-pop: LHR62-C4
x-amz-cf-id: U5cR0f5Vnu4lb17l8zReGXZdCo0cnaHeMSPzqK7ww5n2rU6pXYvkdg==

Gun 1.3.2 unexpectedly returns a 400 error, both in http2 and http mode

iex(1)> {:ok, conn} = :gun.open('partners.betvictor.mobi', 443, %{protocols: [:http]})
{:ok, #PID<0.1185.0>}
iex(2)> {:ok, protocol} = :gun.await_up(conn)
{:ok, :http}
iex(3)> stream = :gun.get(conn, '/en/sport/364/meetings')
#Reference<0.4270986501.1222639619.225254>
iex(4)> flush
{:gun_response, #PID<0.1185.0>, #Reference<0.4270986501.1222639619.225254>,
 :nofin, 400,
 [
   {"content-type", "text/plain; charset=utf-8"},
   {"content-length", "11"},
   {"connection", "keep-alive"},
   {"date", "Tue, 10 Mar 2020 16:32:09 GMT"},
   {"x-cache", "Error from cloudfront"},
   {"via", "1.1 54c4a3ab55229e407630e7a126ca0932.cloudfront.net (CloudFront)"},
   {"x-amz-cf-pop", "CPH50-C1"},
   {"x-amz-cf-id", "afFdwfuuiq3j90Zg5QMJ77QZDzk7xs3J3Nk9diOW9RUYGm6l5I2Bcg=="}
 ]}
{:gun_data, #PID<0.1185.0>, #Reference<0.4270986501.1222639619.225254>, :fin,
 "Bad Request"}
:ok

As I said, I could not make this request work on Hackney either, due to possibly SNI issues - https://github.com/benoitc/hackney/issues/612

Any help in making this work, like curl does?

essen commented 4 years ago

It's just user agent detection.

7> gun:get(C, "/en/sport/364/meetings", #{<<"user-agent">> => <<"curl/7.68.0">>}).     
#Ref<0.2748838253.3639607299.118217>
8> flush().
Shell got {gun_response,<0.110.0>,#Ref<0.2748838253.3639607299.118217>,nofin,
                        200,
essen commented 4 years ago

Note that if you provide a user-agent like <<"Gun">> or <<"Gun/1.0">> you get a 502 instead.