lexiforest / curl_cffi

Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
https://curl-cffi.readthedocs.io/
MIT License
2.56k stars 269 forks source link

Exception while setting custom ja3 parameter on requests.get() for certain cipher suite integer values #443

Open charliedelta02 opened 6 days ago

charliedelta02 commented 6 days ago

Please check the following items before reporting a bug, otherwise it may be closed immediately.

Describe the bug

Cipher Exception:

Exception while setting custom ja3 parameter on requests.get() for certain cipher suite integer values The exception observed is:

[2024-11-26 11:25:42,203] ERROR in curl_cffi_script: 49324
[2024-11-26 11:25:42,209] ERROR in curl_cffi_script:   File "/Users/Documents/scripts/curl_cffi_script.py", line 258, in crawl_url
    response = await s.get(
  File "/Users/python_virtual_env/python3/lib/python3.9/site-packages/curl_cffi/requests/session.py", line 1253, in request
    req, buffer, header_buffer, q, header_recved, quit_now = self._set_curl_options(
  File "/Users/python_virtual_env/python3/lib/python3.9/site-packages/curl_cffi/requests/session.py", line 654, in _set_curl_options
    self._set_ja3_options(c, ja3, permute=permute)
  File "/Users/python_virtual_env/python3/lib/python3.9/site-packages/curl_cffi/requests/session.py", line 304, in _set_ja3_options
    cipher_name = TLS_CIPHER_NAME_MAP[cipher_id]

Cipher 49324 translates to a Hex value: 0xC0,0xAC which is the cipher TLS_ECDHE_ECDSA_WITH_AES_128_CCM as per https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4

Similar issue observed with Cipher values: 65, 64, 49202, 49167, 49194, 106 and a few more.

To Reproduce

async with AsyncSession() as s:
    # This is the ja3n text for Safari v18 on a Mac obtained from: https://tls.browserleaks.com/json
    ja3n_str = "771,49199-49171-49191-49200-49172-49192-156-47-60-157-53-61-65-132-49195-49324-49326-49161-49187-49196-49325-49327-49162-49188-158-51-103-159-57-107-69-136-4865-4866-255,0-10-11-13-23-43-51,23-29-24-256-257-258,0"
    # Another ja3n string that fails
    # ja3n_str = "771,4866-4865-4867-49196-49195-52393-49200-52392-49199-159-52394-163-158-162-49188-49192-49187-49191-107-106-103-64-49198-49202-49197-49201-49190-49194-49189-49193-49162-49172-49161-49171-57-56-51-50-49157-49167-49156-49166-157-156-61-60-53-47-255,0-5-10-11-13-17-23-43-45-50-51,29-23-24-25-30-256-257-258-259-260,0"
    response = await s.get(
        url,
        timeout=30,
        ja3=ja3n_str
    )

Expected behavior The valid custom ja3n fingerprint string must be used upon requests.get().

Versions

Additional context

lexiforest commented 1 day ago

These values are not available in boringssl, thus there is no way to use them unless we patch boringssl, which requires a significant amount of investigation. Please expect an extend time to fix.