Open fakirpic opened 1 year ago
A little late, but here is my current solution
const undici = require("undici")
const tls = require("tls")
// From https://httptoolkit.com/blog/tls-fingerprinting-node-js/
const defaultCiphers = tls.DEFAULT_CIPHERS.split(':');
const shuffledCiphers = [
defaultCiphers[1],
defaultCiphers[2],
defaultCiphers[0],
...defaultCiphers.slice(3)
].join(':');
const connector = undici.buildConnector({ ciphers: shuffledCiphers })
const client = new undici.Client("https://en.zalando.de", { connect: connector })
undici.request("https://en.zalando.de/api/navigation", {
dispatcher: client,
headers: {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
}
}).then(async (res) => {
const body = await res.body.json()
console.log(body)
})
We can specify a custom connector (for tls connections) in the Client constructor. This connector comes from buildConnector, which will pass any arguments it is given to tls.connect. tls.connect accepts a custom list of ciphers using the ciphers
option.
Many web firewall applications like Cloudflare implemented new anti DDoS system based on TLS Fingerprinting, they are whitelisting of up-to-date browsers , we can not send http/s requests to servers behind of such new style firewalls , because we are identified as like DDoS bots, while most of us are just trying to scrape or send API requests.
there is also a very well documented nice blog post that i suggest you to read if you want to learn deeper reasons and results about it
https://httptoolkit.com/blog/tls-fingerprinting-node-js/
Currently, there is no a comfortable way to impersonate a browser's fingerprint in nodejs,
you can checkout your tool's or browser's fingerprint from this link https://check.ja3.zone/
there is a already solution for curl here
https://github.com/lwthiker/curl-impersonate
a Golang based solution for nodejs which i didn't like
https://github.com/depicts/got-tls
if there could be a lower-level way to impersonate browsers tls fingerprints in nodejs ,
i can
ciphers: 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384',
in request package but which is not helping yet to mimic browsers.
i personally would be very very glad, don't know what community thinks.
best regards