Libcurl certificate compression & more ?

[momala454]

I'm using libcurl and php for curl-impersonate, i'm setting the ciphers, the ssl version, disable NPN, but i'm still missing the extensions 27 (compress_certificate) and 17513 (extensionApplicationSettings (boringssl))

I can't set the parameters "--alps" and "--cert-compression brotli" using libcurl and PHP. The curl php extension doesn't support new parameters names (see https://github.com/php/php-src/blob/master/ext/curl/interface.c#L2306 ) What should I do to enable those 2 extensions ?

edit: calling putenv('CURL_IMPERSONATE=chrome98'); or CURL_IMPERSONATE=chrome98 php /path/to/script.php beforehand doesn't help, the two tls extensions are still not enabled.

My guess is that that both "alps" and "cert-compression" have not been yet added to the default parameters on curl-impersonate for libcurl.

Using CURL_IMPERSONATE=chrome98 curl "http://...." is correctly adding the 2 extensions however

[momala454]

nevermind, it was actually my fault, i've used curl_reset, which remove the parameters set using CURL_IMPERSONATE=chrome98

[lwthiker]

Alright, you also need to launch with the LD_PRELOAD env var to replace the default libcurl with libcurl-impersonate

[momala454]

i've modified the library file system wide so I don't need to prepend each execution of my script with LD_PRELOAD.

curl_reset is really useful because i'm reusing my connection between requests, and some parameters set on a previous request might not be useful on a following request. Do you have an idea how to reset without resetting the settings that I can't configure from php ? Because i'm not sure how to properly reset everthing to their default value, like CURLOPT_COOKIEJAR or CURLOPT_PROXY, but I set many parameters, not only those two. That's why curl_reset is useful for me

maybe curl_easy_reset could reset to the default curl_impersonate values ? https://github.com/curl/curl/blob/2610142139d14265ed9acf9ed83cdf73d6bb4d05/lib/easy.c#L959

[lwthiker]

I'll have to look how to change curl_easy_reset() so that the curl-impersonate options are preserved. Probably an internal call to curl_easy_impersonate() right after will do the trick but this has to be checked more thoroughly.

In the meantime, I guess you could just cleanup the handle and create a new one. Does curl_reset() actually preserve the connection or just saves you from destroying the old handle and creating a new one?

[momala454]

curl_reset keep the connection, so doing mutliple requests on the same website is much faster than closing and reiniting curl for each request

[lwthiker]

I see. Then patching curl_easy_reset() is probably the way to go. If you wanna give it a shot you are welcome - probably need to add something similar to this in the reset function. If not I'll have a look at this later.

[lwthiker]

@momala454 You may want to try https://github.com/lwthiker/curl-impersonate/pull/44. curl_easy_reset() is now patched as well, so that the necessary curl options for impersonation are retained (currently for the Chrome version only). I did some manual testing and it looks OK. Please test with your PHP app and let me know if it fixes the issue for you.

[momala454]

thank you, i will test this next week

[momala454]

i have a small problem with your pull request (actually, with the whole libcurl version). When i'm using CURL_IMPERSONATE=chrome98, i'm unable to overwrite the default user-agent header to Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36 it stays with Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36

when I set CURLOPT_USERAGENT, it does not overwrite the default user-agent set with "CURL_IMPERSONATE" env variable. It only reset it if I modify the header "user-agent" using CURLOPT_HTTPHEADER

beside this small thing, the fix is working fine thank you

[lwthiker]

Got it, I'll open a separate issue for this. This one will be closed once I merge #44