lexiforest / curl_cffi

Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
https://curl-cffi.readthedocs.io/
MIT License
2.38k stars 257 forks source link

HTTP/2 stream not closed cleanly #79

Closed mavic111 closed 1 year ago

mavic111 commented 1 year ago

To reproduce error:

from curl_cffi import requests

client = requests.Session(impersonate="chrome110")

res = client.get('https://google.com')
print("Google", res.status_code)
res = client.get('https://shopee.co.id')
print("Shopee", res.status_code)
Google 200
Traceback (most recent call last):
  File "C:\Users\muham\source\repos\blegping\venv\Lib\site-packages\curl_cffi\requests\session.py", line 436, in request
    c.perform()
  File "C:\Users\muham\source\repos\blegping\venv\Lib\site-packages\curl_cffi\curl.py", line 191, in perform
    self._check_error(ret, action="perform")
  File "C:\Users\muham\source\repos\blegping\venv\Lib\site-packages\curl_cffi\curl.py", line 84, in _check_error
    raise error
curl_cffi.curl.CurlError: Failed to perform, ErrCode: 92, Reason: 'HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\muham\source\repos\blegping\main.py", line 7, in <module>
    res = client.get('https://shopee.co.id')
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\muham\source\repos\blegping\venv\Lib\site-packages\curl_cffi\requests\session.py", line 438, in request
    raise RequestsError(e)
curl_cffi.requests.errors.RequestsError: Failed to perform, ErrCode: 92, Reason: 'HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)'
perklet commented 1 year ago

╰─>$ cat test.py                                               
from curl_cffi import requests

client = requests.Session(impersonate="chrome110")

res = client.get('https://google.com')
print("Google", res.status_code)
res = client.get('https://shopee.co.id')
print("Shopee", res.status_code)

╰─>$ python test.py                                            
Google 200
Shopee 200

Sorry, I can not reproduce this. Network problems are not always reproducible, especially http2 related ones.

This should be resolved if you force libcurl to use http1.1 only. Unfortunately, I did not expose the interface for setting this option easily, which should be fixed in the next minor version. For now, you can add one line here to use http1.1 explicitly.

CURL_HTTP_VERSION_1_1 = 2
c.setopt(CurlOpt.HTTP_VERSION, CURL_HTTP_VERSION_1_1)
mavic111 commented 1 year ago

it works, thanks.

perklet commented 1 year ago

For v0.5.8 above, use this:

from curl_cffi import requests, CurlHttpVersion

requests.get("https://example.com", http_version=CurlHttpVersion.V1_1)