lexiforest / curl_cffi

Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
https://curl-cffi.readthedocs.io/
MIT License
2.49k stars 265 forks source link

[BUG] Encoding header values error #354

Closed AnasBoujamaa closed 4 months ago

AnasBoujamaa commented 4 months ago

Describe the bug When I send headers where the values have special characters like 'Ñ' I get this error (this doesn't happen with libraries like requests):

Traceback (most recent call last):
  File "/home/anas/test.py", line 7, in <module>
    response = requests.post('https://httpbin.org/anything', headers=data)
  File "/home/anas/.local/lib/python3.10/site-packages/curl_cffi/requests/__init__.py", line 125, in request
    return s.request(
  File "/home/anas/.local/lib/python3.10/site-packages/curl_cffi/requests/session.py", line 900, in request
    req, buffer, header_buffer, q, header_recved, quit_now = self._set_curl_options(
  File "/home/anas/.local/lib/python3.10/site-packages/curl_cffi/requests/session.py", line 397, in _set_curl_options
    h.update(headers)
  File "/home/anas/.local/lib/python3.10/site-packages/curl_cffi/requests/headers.py", line 249, in update
    headers = Headers(headers)
  File "/home/anas/.local/lib/python3.10/site-packages/curl_cffi/requests/headers.py", line 112, in __init__
    self._list = [
  File "/home/anas/.local/lib/python3.10/site-packages/curl_cffi/requests/headers.py", line 116, in <listcomp>
    normalize_header_value(v, encoding),
  File "/home/anas/.local/lib/python3.10/site-packages/curl_cffi/requests/headers.py", line 94, in normalize_header_value
    return value.encode(encoding or "ascii")
UnicodeEncodeError: 'ascii' codec can't encode character '\xf1' in position 0: ordinal not in range(128)

To Reproduce

from curl_cffi import requests

data = {
    'Example': 'ñ',
}

response = requests.post('https://httpbin.org/anything', headers=data)
print(response.json())

Expected behavior httpbin returning that I sent those exact headers.

Versions