search

apify / fingerprint-suite

Browser fingerprinting tools for anonymizing your scrapers. Developed by Apify.
Apache License 2.0
1k stars 104 forks source link

Fingerprint Generator to initially generate only the user agent? #297

Closed Biswajit96 closed 5 months ago

Biswajit96 commented 5 months ago

Could you clarify if it's possible to use the Apify Fingerprint Generator to initially generate only the user agent, perform some tests, and then generate the full fingerprint for that user agent if needed?

The reason for this request is that the Apify Fingerprint Generator is consuming a lot of CPU. Generating just the user agent first would use less CPU, allowing me to test efficiently. If the tests pass, I would then generate the complete fingerprint and proceed.

barjin commented 5 months ago

Hello and thank you for your interest in this project.

In fact, the fingerprint-generator first generates a set of HTTP headers (including the user-agent string) using the header-generator package and only then goes on to generate other stuff (like screen resolution, GPU vendor etc.). This happens in the getFingerprint() method here:

https://github.com/apify/fingerprint-suite/blob/f512962cd84a6fb6df27adf91c20b19e260970e4/packages/fingerprint-generator/src/fingerprint-generator.ts#L159-L163

Since this project is open-source, you can easily hack this (e.g. first generate your HTTP headers with the header-generator and then pass the user-agent as a new parameter for the fingeprint-generator - overriding the current generation on line 159).

What I'm more curious about - you're saying the fingerprint-generator package is using too much CPU. This is something we haven't experienced yet. Can you please provide some more details about your setup?

Cheers!