fabienvauchelles / scrapoxy

Scrapoxy is a super proxy aggregator, allowing you to manage all proxies in one place 🎯, rather than spreading it across multiple scrapers 🕸️. It also smartly handles traffic routing 🔀 to minimize bans and increase success rates 🚀.
http://scrapoxy.io
MIT License
1.94k stars 232 forks source link

Adding browser specific TLS fingerprints #247

Open Samy-ib opened 1 month ago

Samy-ib commented 1 month ago

Hello I just discovered Scrapoxy and I'am already in love (even though I'am still waiting for Oxylabs support). Good job on adding the cipher suite shuffling feature.

But I would like to ask if it's possible to improve on that by adding the possibility to use the exact TLS fingerprint of popular browsers. There is a GO library that allows to do just that (https://github.com/refraction-networking/utls). As I'am an absolute noob with TS and don't know if there is a way to use some GO stuff on Scrapoxy or if there some other way to archive that.

Another suggestion would be adding HTTP2 fingerprint mimic of popular browser.

Both of those features can help when it comes to anti bot detection.

If Scrapoxy adds those two features it will become a godlike tool.

Would like to hear what you think of it and it's feasibility.

Best reagards.

fabienvauchelles commented 1 month ago

Hi @Samy-ib ,

Thank you for using Scrapoxy!

Oxylabs:

I’m interested in implementing the Oxylabs connector. However, I prefer when providers reach out to me directly, as it tends to build a stronger relationship. Could you open a ticket with Oxylabs to request support for Scrapoxy? It would help me a lot!!!

TLS Fingerprint:

I’m unable to use the bogdanfinn tls-client library, despite its strengths in GO and support for HTTP2 fingerprinting (note: this library is build on top of uTLS). Unfortunately, it doesn't support Node.js streams, which are essential for Scrapoxy and avoid memory outage.

To implement TLS or HTTP2 fingerprint support, I would need to rewrite the C++ portion in Node.js—a challenging task that requires significant effort. This is something I plan to work on, but I don't have an ETA yet.