nodejs / undici

An HTTP/1.1 client, written from scratch for Node.js
https://nodejs.github.io/undici
MIT License
5.92k stars 512 forks source link

Make TLS Fingerprint great again #1983

Open fakirpic opened 1 year ago

fakirpic commented 1 year ago

Many web firewall applications like Cloudflare implemented new anti DDoS system based on TLS Fingerprinting, they are whitelisting of up-to-date browsers , we can not send http/s requests to servers behind of such new style firewalls , because we are identified as like DDoS bots, while most of us are just trying to scrape or send API requests.

there is also a very well documented nice blog post that i suggest you to read if you want to learn deeper reasons and results about it

https://httptoolkit.com/blog/tls-fingerprinting-node-js/

Currently, there is no a comfortable way to impersonate a browser's fingerprint in nodejs,

you can checkout your tool's or browser's fingerprint from this link https://check.ja3.zone/

there is a already solution for curl here

https://github.com/lwthiker/curl-impersonate

a Golang based solution for nodejs which i didn't like

https://github.com/depicts/got-tls

if there could be a lower-level way to impersonate browsers tls fingerprints in nodejs ,

i can

ciphers: 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384',

in request package but which is not helping yet to mimic browsers.

i personally would be very very glad, don't know what community thinks.

best regards

noahcoolboy commented 2 months ago

A little late, but here is my current solution

const undici = require("undici")
const tls = require("tls")

// From https://httptoolkit.com/blog/tls-fingerprinting-node-js/
const defaultCiphers = tls.DEFAULT_CIPHERS.split(':');
const shuffledCiphers = [
    defaultCiphers[1],
    defaultCiphers[2],
    defaultCiphers[0],
    ...defaultCiphers.slice(3)
].join(':');

const connector = undici.buildConnector({ ciphers: shuffledCiphers })
const client = new undici.Client("https://en.zalando.de", { connect: connector })

undici.request("https://en.zalando.de/api/navigation", {
    dispatcher: client,
    headers: {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
    }
}).then(async (res) => {
    const body = await res.body.json()
    console.log(body)
})

We can specify a custom connector (for tls connections) in the Client constructor. This connector comes from buildConnector, which will pass any arguments it is given to tls.connect. tls.connect accepts a custom list of ciphers using the ciphers option.