bogdanfinn / tls-client

net/http.Client like HTTP Client with options to select specific client TLS Fingerprints to use for requests.
BSD 4-Clause "Original" or "Old" License
670 stars 134 forks source link

browsers doesnt reproduce updated tls specs #10

Closed Harusakii closed 1 year ago

Harusakii commented 1 year ago

Issue:

When using the chrome browser and request tls.peet.ws/api/all, it shows me this ja3 string

(771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-10-11-13-16-23-43-51-65281-45-21,29-23-24,0)

and in the extensions, this is included:

{
        "name": "padding (21)",
        "padding_data_length": 514
      }

now when using that same ja3 and building a custom Client with it, the same site shows me everything perfectly fine, except it doesnt show me the padding Extension, its just gone in the extensions list and the shown ja3 string

(771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-10-11-13-16-23-43-51-65281-45,29-23-24,0)

I noticed that when I add other extensions like i.e. 17513 or 27, it then also shows the padding extension again. But then again, its not the same as my actual browser :/

bogdanfinn commented 1 year ago

hey @Harusakii what chrome do you actually use? because when i use my latest chrome browser i get another ja3 than you.

Also please make sure that you visit tls.peet.ws/api/all in a fresh session (for example incognitor mode) because the output might change due to the session tickets.

so for example when i visit tls.peet.ws/api/all in a fresh browser session and then just reload the page i get two different ja3 strings.

so i'm just curious what chrome browser you are actually using because mine 106 has the following ja3: 771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21,29-23-24,0

FYI peet also open sourced his api: https://github.com/wwhtrbbtt/TrackMe You will notice there for example that he removes on purpose the Grease Extension as it would change the ja3 on every request.

So if you want to have the real client hello of a client do not rely fully on his api but use tools like wireshark.

Harusakii commented 1 year ago

Sorry for not responding for a while.

And yes I figured it cant be always the same, but for me every ja3 is different to what the presets we have in the utls. like for example, my firefox ja3 is a bit different (done in incognito mode, rechecked with wireshark, its also different there):

(my Firefox Browser in wireshark client hello) Transmission Control Protocol, Src Port: 57477, Dst Port: 443, Seq: 1, Ack: 1, Len: 517 Transport Layer Security TLSv1.3 Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: TLS 1.0 (0x0301) Length: 512 Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 508 Version: TLS 1.2 (0x0303) Random: 18573c0ff30a50746d8b3c9a7179c163f10dd5df3888b0d00bfec9869857f159 Session ID Length: 32 Session ID: 2a98793c783ca5265118801e246ed9174850db6e36c5123885b9c5e692df6a89 Cipher Suites Length: 34 Cipher Suites (17 suites) Compression Methods Length: 1 Compression Methods (1 method) Extensions Length: 401 Extension: server_name (len=16) Extension: supported_groups (len=8) Extension: ec_point_formats (len=2) Extension: signature_algorithms (len=22) Extension: application_layer_protocol_negotiation (len=14) Extension: extended_master_secret (len=0) Extension: supported_versions (len=5) Extension: key_share (len=107) Extension: renegotiation_info (len=1) Extension: padding (len=186) [JA3 Fullstring: 771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-156-157-47-53,0-10-11-13-16-23-43-51-65281-21,29-23-24,0] [JA3: 4a2366515eb6384bffa29b1061b22c9c]

(the firefox 106 preset in wireshark): Transmission Control Protocol, Src Port: 57484, Dst Port: 443, Seq: 1, Ack: 1, Len: 517 Transport Layer Security TLSv1.3 Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: TLS 1.0 (0x0301) Length: 512 Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 508 Version: TLS 1.2 (0x0303) Random: 70f0884f2e70017e0f5e27de7f75a4a98811f9fe77e00835f7215f8bb5c11673 Session ID Length: 32 Session ID: 1d2f6c5f1fc779e83ac525d81f98dc53564b4daf9c527f5c5d91480b501e99c7 Cipher Suites Length: 34 Cipher Suites (17 suites) Compression Methods Length: 1 Compression Methods (1 method) Extensions Length: 401 Extension: server_name (len=16) Extension: extended_master_secret (len=0) Extension: renegotiation_info (len=1) Extension: supported_groups (len=14) Extension: ec_point_formats (len=2) Extension: session_ticket (len=0) Extension: application_layer_protocol_negotiation (len=14) Extension: status_request (len=5) Extension: delegated_credentials (len=10) Extension: key_share (len=107) Extension: supported_versions (len=5) Extension: signature_algorithms (len=24) Extension: psk_key_exchange_modes (len=2) Extension: record_size_limit (len=2) Extension: padding (len=139) [JA3 Fullstring: 771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-34-51-43-13-45-28-21,29-23-24-25-256-257,0] [JA3: 579ccef312d18482fc42e2b822ca2430]

I really dont know why, I also tried reinstalling but it keeps getting the same result. The same happening with Chrome.

EDIT: Tbh I think somethings wrong w my browsers lol, cause at other people, it is actually the same as yours, so I am wondering why i aint getting the same results. Would be cool if I get some help on that

bogdanfinn commented 1 year ago

@Harusakii sorry for not responding yet. As you are closing the issue i guess you found the reason?

my best guess would be to check the following things:

i do not have any clue actually why your browser would change the fingerprint on every request ... sounds weird. Which site are you actually testing against?

Harusakii commented 1 year ago

@Harusakii sorry for not responding yet. As you are closing the issue i guess you found the reason?

my best guess would be to check the following things:

  • Network configuration like VPN / Proxies / etc.
  • Browser Extensions maybe
  • Exact Browser Version (maybe you are using unstable / beta versions and not official releases?)

i do not have any clue actually why your browser would change the fingerprint on every request ... sounds weird. Which site are you actually testing against?

Well I didnt find out what it causes, so I will trust your specs that they are up-to date. Also I didnt use any Proxy, VPN or sth else to test it, no browser extensions and I am using the official builds of the browsers (Chrome 107.0.5304.107, Firefox 106.0.5).

The Client Hello is pretty much the same for whatever site I am trying it against, so yeah. But as its not a problem with your TLS client, I closed the issue here. Maybe once I find out the reason I will post an update here.

bogdanfinn commented 1 year ago

@Harusakii afaik there is a feature of random order in tls extensions in some future google chrome browser.

Harusakii commented 1 year ago

Well, here we are four months later, and I fricking found the problem (while being out with friends, they said something to get me to think about this), and I never thought about my antivirus LOL.

so yeah, the problem was my antivirus, that changes TLS specs. So to all future people having this problem, truly look at every background process if it doesnt work lol.