search

daijro / camoufox

🦊 Undetected web scraping browser
https://pypi.org/project/camoufox
Mozilla Public License 2.0
90 stars 8 forks source link

User-Agent & proxy detection: BrowserScan.net, Pixelscan.net #14

Closed satanmp closed 2 weeks ago

satanmp commented 2 weeks ago

image image

daijro commented 2 weeks ago

Hello,

The detection of proxies isn't done at the browser level, it's done at the network level. Unfortunately there isn't anything that can be done in Camoufox that can fix it, besides switching to a provider that doesn't cause flags.

The User Agent detection works by checking for JS behavior that is specific to Firefox 130, and comparing it to the claimed userAgent (in this case 125). In my testing, I've only seen this specific to BrowserScan.net, and it doesn't seem to be used by WAFs in production at the moment (maybe in the future).

Because having a broad range of User Agents is helpful in avoiding detection for most other websites, I've decided against locking the userAgent version in order to get around this edge case. In the future, I do plan on locking it.

satanmp commented 2 weeks ago

Hello,

The detection of proxies isn't done at the browser level, it's done at the network level. Unfortunately there isn't anything that can be done in Camoufox that can fix it, besides switching to a provider that doesn't cause flags.

The User Agent detection works by checking for JS behavior that is specific to Firefox 130, and comparing it to the claimed userAgent (in this case 125). In my testing, I've only seen this specific to BrowserScan.net, and it doesn't seem to be used by WAFs in production at the moment (maybe in the future).

Because having a broad range of User Agents is helpful in avoiding detection for most other websites, I've decided against locking the userAgent version in order to get around this edge case. In the future, I do plan on locking it.

thanks a lot...，Thank you for your efforts