sindresorhus / got

🌐 Human-friendly and powerful HTTP request library for Node.js
MIT License
14.27k stars 935 forks source link

Request hanged unexpectedly when user-agent header supplied #1876

Closed NeverBehave closed 3 years ago

NeverBehave commented 3 years ago

Describe the bug

Actual behavior

When requesting the site https://www.mckinsey.com.cn with any user-agent, got will hang

no error message generated

Removing user-agent or adding different header won't cause the issue

Expected behavior

A response should be returned

Code to reproduce

https://runkit.com/neverbehave/6152cbc94fd89300080351dc

I am not sure about the cause, and there is no retry-after header or something strange, based on curl result:

$ curl https://www.mckinsey.com.cn/ -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36' -v -o /dev/null
* Connected to www.mckinsey.com.cn (173.222.162.145) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [233 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [102 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2668 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [148 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=CN; ST=Beijing; L=Beijing; O=McKinsey & Consulting Company Inc., Shanghai Beijing Branch; CN=solutions.mckinsey.com.cn
*  start date: Mar 13 00:00:00 2021 GMT
*  expire date: Mar 23 23:59:59 2022 GMT
*  subjectAltName: host "www.mckinsey.com.cn" matched cert's "www.mckinsey.com.cn"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7f8a0b80a400)
> GET / HTTP/2
> Host: www.mckinsey.com.cn
> Accept: */*
> user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< content-type: text/html; charset=UTF-8
< link: <https://www.mckinsey.com.cn/wp-json/>; rel="https://api.w.org/"
< link: <https://www.mckinsey.com.cn/wp-json/wp/v2/pages/13297>; rel="alternate"; type="application/json"
< link: <https://www.mckinsey.com.cn/>; rel=shortlink
< strict-transport-security: max-age=31536000; includeSubDomains
< x-akamai-transformed: 9 70242 0 pmb=mTOE,2mRUM,1
< expires: Tue, 28 Sep 2021 08:18:04 GMT
< cache-control: max-age=0, no-cache, no-store
< pragma: no-cache
< date: Tue, 28 Sep 2021 08:18:04 GMT
< server-timing: cdn-cache; desc=MISS
< server-timing: edge; dur=572
< server-timing: origin; dur=254
< set-cookie: bm_mi=83EE8AEE13CB8255A62C532C401CBD60~7sbiGqKe87aAVOrXeRjZRFujUakZLUDTRzZ7n8uh7fZLZg1uZMpOm7g97fg0ucHGS4Da7MadBDU1+fEdbPQS7D62E0IRmqPbs/aXimR4NFa7K4sy+6nIesedZMEWE0orb0J1lRVkcqoZzRy0BJjr89y+Rpdosp2Uxnmts0GhvFpUhb0H+hNh682P18cqV/n21jFjogGcbEsd8j3shFci6BS8iK484FwOErOdckAnCcs=; Domain=.mckinsey.com.cn; Path=/; Max-Age=0; HttpOnly
< set-cookie: ak_bmsc=A8FB9AA30DB48A23767D5F6B7E8328BF~000000000000000000000000000000~YAAQkSwtF4c2dOR7AQAAX3h7Kw0DXwrRl+RUrTxHM9NW9cKE3ZtBh+tn+vEHz6M3NlNVHjeYm4dWF5+Clg8e5nOYnH59rXyE012ZgsH49C+ZLJ5/NFHkyirMVZTR4XwRTnh81g4QJsAOWpT/WCT2KWL+Z5n5OgmSNt1ESlfQvaz7qzvi/VO4TC7cIOP6bFFwhvr7DfD68gmH1DTElgPf9uUQjoDMwD8mI5iaGIcI0631ZPd9pkwsmVgplnjLuwou3RYCSaZ5F5SYjXo/eGEqJH3XDjjSdWqBgchhrAu39g4Z7ubiwcYqipG1i18F9ajkjPzoLEz0aLFjLGPPQ5lKQnmKW29Jy7xamHjKElM9CrnysZ7xRp+zWAaU2NenWIRUIOLIoT7WGJ3mb8izNg==; Domain=.mckinsey.com.cn; Path=/; Expires=Tue, 28 Sep 2021 10:18:03 GMT; Max-Age=7199; HttpOnly
<

Checklist

version: 11.8.2

szmarczak commented 3 years ago

This is an issue with the server. curl hangs as well:

curl https://www.mckinsey.com.cn -H 'user-agent: asdf' -H 'accept-encoding: gzip, deflate, br' -v --http1.1

Disabling decompress seems to mitigate the issue.