lexiforest / curl_cffi

Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
https://curl-cffi.readthedocs.io/
MIT License
2.52k stars 266 forks source link

Response 400 Bad request : Your browser sent an invalid request #289

Closed Lukas-drz closed 7 months ago

Lukas-drz commented 7 months ago

When trying to request a datadome protected website leboncoin.fr, I'm not blocked by the anti-bot with 403 Forbidden but with curl_cffi I have this weird 400 Bad request : Your browser sent an invalid request, do you know why I have this one ? I tried using simple requests and sessions, both result in the same thing. Some requests may go well, but others not. I use the impersonate="chrome119" or "chrome120" The website should reply with a response 200. I'm using 0.6.2 and Ubuntu 22.04 I'm using async but it worked like that with request but sometimes I had response 403, never 400. Either the response headers are empty or : Headers({'content-type': 'text/html', 'content-length': '90', 'date': 'Fri, 05 Apr 2024 21:04:27 GMT', 'cache-control': 'no-cache', 'x-cache': 'Error from cloudfront', 'via': '1.1 ee4db0d243ceb0d1993e5f46ad6c0f00.cloudfront.net (CloudFront)', 'x-amz-cf-pop': 'CDG50-P> Thanks for helping

perklet commented 7 months ago

Sorry, I can not reproduce the error.

>>> from curl_cffi import requests
>>> requests.get("https://leboncoin.fr").text[:100]
'<html><head><title>leboncoin.fr</title><style>#cmsg{animation: A 1.5s;}@keyframes A{0%{opacity:0;}99'
>>> requests.get("https://leboncoin.fr", impersonate="chrome120").text[:100]
'<!DOCTYPE html><html lang="fr"><head><meta charSet="utf-8"/><link type="application/opensearchdescri'

Feel free to reopen with detailed code snippets.