lexiforest / curl_cffi

Python binding for curl-impersonate via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
https://curl-cffi.readthedocs.io/
MIT License
2.1k stars 249 forks source link

Not passing "impersonate" #240

Closed 64chevy closed 8 months ago

64chevy commented 8 months ago

If I don't pass any impersonate argument in the request, what does it do by default?

perklet commented 8 months ago

The you got standard curl headers:

>>> from curl_cffi import requests                                  
>>> print(requests.get("https://httpbin.org/headers").text)         
{                                                                   
  "headers": {                                                      
    "Accept": "*/*",                                                
    "Accept-Encoding": "gzip, deflate, br",                         
    "Host": "httpbin.org",                                          
    "X-Amzn-Trace-Id": "Root=1-65c045a8-6af160e4183430ef1163c738"   
  }                                                                 
}                                                                   

>>> print(requests.get("https://tls.browserleaks.com/json").text)   
{                                                                   
  "user_agent": "",                                                 
  "ja3_hash": "ff778db1da7ed3e722c06cd7fbd90f04",                   
  "ja3_text": "771,4865-4866-4867-49195-49199-49196-49200-52393-5239
4,0",                                                               
  "ja3n_hash": "e664635a7b3e64ea1f24e582688ac4c6",                  
  "ja3n_text": "771,4865-4866-4867-49195-49199-49196-49200-52393-523
24,0",                                                              
  "akamai_hash": "52d84b11737d980aef856699f885ca86",                
  "akamai_text": "1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p"
}                                                                   

vs curl

╰─>$ curl https://httpbin.org/headers                            
{                                                                
  "headers": {                                                   
    "Accept": "*/*",                                             
    "Host": "httpbin.org",                                       
    "User-Agent": "curl/8.1.2",                                  
    "X-Amzn-Trace-Id": "Root=1-65c045f8-40919b7827bb9ecb4d3f996c"
  }                                                              
}                                                                

╰─>$ curl https://tls.browserleaks.com/json                      
{                                                                
  "user_agent": "curl/8.1.2",                                    
  "ja3_hash": "2bab0327a296230f9f6427341e716ea0",                
  "ja3_text": "771,4866-4867-4865-49200-49196-49192-49188-49172-4
190-69-156-60-47-186-65-49169-49159-5-4-49170-49160-22-10-255,43-
  "ja3n_hash": "89c23ef504c3cc9ccd6580643494edae",               
  "ja3n_text": "771,4866-4867-4865-49200-49196-49192-49188-49172-
-190-69-156-60-47-186-65-49169-49159-5-4-49170-49160-22-10-255,0-
  "akamai_hash": "64a832f547be33249bf4d33e8a46c5dc",             
  "akamai_text": "3:100;4:10485760;2:0|1048510465|0|m,s,a,p"     
}                                                                

Notice the tls/h2 fingerprints were altered but not the same as browsers.