lexiforest / curl_cffi

Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
https://curl-cffi.readthedocs.io/
MIT License
2.49k stars 265 forks source link

[BUG] Unsupported ctypes for getinfo to work #407

Open ringus1 opened 1 month ago

ringus1 commented 1 month ago

Describe the bug Some CurlInfo constants can't be retrieved seemingly because types are not supported, such as curl_off_t.

Raw CURL command

> curl --compressed -so /dev/null http://www.whatsmyip.org/http-compression-test/ -w '%{size_download}'
256

Python attempt

> import curl_cffi

> curl = curl_cffi.Curl()
> curl.setopt(curl_cffi.CurlOpt.URL, "http://www.whatsmyip.org/http-compression-test/")
> curl.setopt(curl_cffi.CurlOpt.ACCEPT_ENCODING, 'gzip')
> curl.perform()

> curl.getinfo(curl_cffi.CurlInfo.SIZE_DOWNLOAD_T)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.12/site-packages/curl_cffi/curl.py", line 243, in getinfo
    c_value = ffi.new(ret_option[option & 0xF00000])
                      ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 6291456

Some more insight

> from curl_cffi._wrapper import ffi, lib
> ffi.getctype("long")
'long'
> ffi.getctype("char")
'char'
> ffi.getctype("curl_off_t")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ffi.error: undefined type name
curl_off_t

Expected behavior CurlInfo options can be retrieved via Curl.getinfo

Versions

lexiforest commented 1 month ago

To expose the curl_off_t type, we need to include typedefs from curl/system.h in ffi.cdef().

usedev commented 1 month ago

curl_off_t looks like a long

curl.getinfo(curl_cffi.CurlInfo.SIZE_DOWNLOAD_T)
ret_option = {
    0x100000: "char**",
    0x200000: "long*",
    0x300000: "double*",
    0x400000: "struct curl_slist **",
    0x500000: "long*",
    0x600000: "long*"
}
ret_cast_option = {
    0x100000: ffi.string,
    0x200000: int,
    0x300000: float,
    0x500000: int,
    0x600000: int,
}
lexiforest commented 1 month ago

I tried to add the typedef to cdef, but it does not work as documented, perhaps it's the more realistic way to do this.

curl_off_t looks like a long

curl.getinfo(curl_cffi.CurlInfo.SIZE_DOWNLOAD_T)
ret_option = {
    0x100000: "char**",
    0x200000: "long*",
    0x300000: "double*",
    0x400000: "struct curl_slist **",
    0x500000: "long*",
    0x600000: "long*"
}
ret_cast_option = {
    0x100000: ffi.string,
    0x200000: int,
    0x300000: float,
    0x500000: int,
    0x600000: int,
}