search

FlorianREGAZ / Python-Tls-Client

Advanced HTTP Library
MIT License
626 stars 130 forks source link

failed to do request: Get "https://tls.peet.ws/api/all": remote error: tls: unexpected message #54

Open wangluozhe opened 1 year ago

wangluozhe commented 1 year ago

success ja3

771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21,29-23-24,0

error ja3

771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21-41,29-23-24,0

test code

config = {'ja3_string': '771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21,29-23-24,0', 'h2_settings': {'HEADER_TABLE_SIZE': 65536, 'MAX_CONCURRENT_STREAMS': 1000, 'INITIAL_WINDOW_SIZE': 6291456, 'MAX_HEADER_LIST_SIZE': 262144}, 'h2_settings_order': ['HEADER_TABLE_SIZE', 'MAX_CONCURRENT_STREAMS', 'INITIAL_WINDOW_SIZE', 'MAX_HEADER_LIST_SIZE'], 'supported_signature_algorithms': ['ECDSAWithP256AndSHA256', 'PSSWithSHA256', 'PKCS1WithSHA256', 'ECDSAWithP384AndSHA384', 'PSSWithSHA384', 'PKCS1WithSHA384', 'PSSWithSHA512', 'PKCS1WithSHA512', 'PKCS1WithSHA1'], 'supported_delegated_credentials_algorithms': None, 'supported_versions': ['GREASE', '1.3', '1.2'], 'key_share_curves': ['GREASE', 'X25519'], 'cert_compression_algo': 'brotli', 'pseudo_header_order': [':method', ':authority', ':scheme', ':path'], 'connection_flow': 15663105, 'priority_frames': None, 'header_order': ['cache-control', 'sec-ch-ua', 'sec-ch-ua-mobile', 'sec-ch-ua-platform', 'upgrade-insecure-requests', 'user-agent', 'accept', 'sec-fetch-site', 'sec-fetch-mode', 'sec-fetch-user', 'sec-fetch-dest', 'accept-encoding', 'accept-language'], 'header_priority': {'weight': 255, 'streamDep': 0, 'exclusive': True}}
session = tls_client.Session(**config)
headers = {
    "cache-control": "max-age=0",
    "sec-ch-ua": '".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"',
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": '"Windows"',
    "upgrade-insecure-requests": "1",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "sec-fetch-site": "none",
    "sec-fetch-mode": "navigate",
    "sec-fetch-user": "?1",
    "sec-fetch-dest": "document",
    # "accept-encoding": "gzip, deflate, br",
    "accept-encoding": "deflate",
    "accept-language": "zh-CN,zh;q=0.9"
}
session.headers.update(headers)
response = session.get(url="https://tls.peet.ws/api/all")
print(response.text)
wangluozhe commented 1 year ago

However, the wrong JA3 value can be used normally in the browser

{
    "ip": "1.203.179.156:51282",
    "http_version": "h2",
    "method": "GET",
    "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
    "tls": {
        "ciphers": [
            "TLS_GREASE (0xBABA)",
            "TLS_AES_128_GCM_SHA256",
            "TLS_AES_256_GCM_SHA384",
            "TLS_CHACHA20_POLY1305_SHA256",
            "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
            "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
            "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
            "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
            "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
            "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
            "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
            "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",
            "TLS_RSA_WITH_AES_128_GCM_SHA256",
            "TLS_RSA_WITH_AES_256_GCM_SHA384",
            "TLS_RSA_WITH_AES_128_CBC_SHA",
            "TLS_RSA_WITH_AES_256_CBC_SHA"
        ],
        "extensions": [
            {
                "name": "TLS_GREASE (0x7a7a)"
            },
            {
                "name": "server_name (0)",
                "server_name": "tls.peet.ws"
            },
            {
                "name": "extended_master_secret (23)",
                "master_secret_data": "",
                "extended_master_secret_data": ""
            },
            {
                "name": "extensionRenegotiationInfo (boringssl) (65281)",
                "data": "00"
            },
            {
                "name": "supported_groups (10)",
                "supported_groups": [
                    "TLS_GREASE (0x7a7a)",
                    "X25519 (29)",
                    "P-256 (23)",
                    "P-384 (24)"
                ]
            },
            {
                "name": "ec_point_formats (11)",
                "elliptic_curves_point_formats": [
                    "0x00"
                ]
            },
            {
                "name": "session_ticket (35)",
                "data": ""
            },
            {
                "name": "application_layer_protocol_negotiation (16)",
                "protocols": [
                    "h2",
                    "http/1.1"
                ]
            },
            {
                "name": "status_request (5)",
                "status_request": {
                    "certificate_status_type": "OSCP (1)",
                    "responder_id_list_length": 0,
                    "request_extensions_length": 0
                }
            },
            {
                "name": "signature_algorithms (13)",
                "signature_algorithms": [
                    "ecdsa_secp256r1_sha256",
                    "rsa_pss_rsae_sha256",
                    "rsa_pkcs1_sha256",
                    "ecdsa_secp384r1_sha384",
                    "rsa_pss_rsae_sha384",
                    "rsa_pkcs1_sha384",
                    "rsa_pss_rsae_sha512",
                    "rsa_pkcs1_sha512"
                ]
            },
            {
                "name": "signed_certificate_timestamp (18)"
            },
            {
                "name": "key_share (51)",
                "shared_keys": [
                    {
                        "TLS_GREASE (0x7a7a)": "00"
                    },
                    {
                        "X25519 (29)": "29ec97a483c73d4ee9ee55a3a74924fec288897b9fc1ce2c1ed06b4042fb5f6f"
                    }
                ]
            },
            {
                "name": "psk_key_exchange_modes (45)",
                "PSK_Key_Exchange_Mode": "PSK with (EC)DHE key establishment (psk_dhe_ke) (1)"
            },
            {
                "name": "supported_versions (43)",
                "versions": [
                    "TLS_GREASE (0x5a5a)",
                    "TLS 1.3",
                    "TLS 1.2"
                ]
            },
            {
                "name": "compress_certificate (27)",
                "algorithms": [
                    "brotli (2)"
                ]
            },
            {
                "name": "application_settings (17513)",
                "protocols": [
                    "h2"
                ]
            },
            {
                "name": "TLS_GREASE (0xfafa)"
            },
            {
                "name": "padding (21)",
                "padding_data_length": 88
            },
            {
                "name": "pre_shared_key (41)",
                "data": "00770071bde576019217d3a3d6796aa940d22cc9fb8c112338f887bda85a4a63bcd64d87a59aa24cf8693ddf33efe3a0ee87f6fa4260332cba65004b649e955ee9fe784239c928cf60fdb8727a1b420e5c9e1a9f47d84f8d8adfb41a07564a11c84da95ca34374a85f8c6bdb33e0c5ed1cc05d8fecc814e25e002120bc215e698a3dd39c65163ec34fba22de72f07d142fac55e2e94eb46486bea0b8"
            }
        ],
        "tls_version_record": "771",
        "tls_version_negotiated": "772",
        "ja3": "771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21-41,29-23-24,0",
        "ja3_hash": "0d69ff451640d67ee8b5122752834766",
        "peetprint (WIP)": "GREASE-772-771|2-1.1|GREASE-29-23-24|1027-2052-1025-1283-2053-1281-2054-1537|1|2|GREASE-4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53|GREASE-0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-GREASE-21-41",
        "peetprint_hash (WIP)": "a0b574f6b401bfa9606352c31a790784",
        "client_random": "ad7c9d9de8952075dfc980682c5dfef674870d37ba123629f01a650e8627f05a",
        "session_id": "fe9c87ccfef5cbf4f8f0e17e0494b851e37a695634235871069029b25420b27b"
    },
    "http2": {
        "akamai_fingerprint": "1:65536,3:1000,4:6291456,6:262144|15663105|0|m,a,s,p",
        "akamai_fingerprint_hash": "7ad845f20fc17cc8088a0d9312b17da1",
        "sent_frames": [
            {
                "frame_type": "SETTINGS",
                "length": 24,
                "settings": [
                    "HEADER_TABLE_SIZE = 65536",
                    "MAX_CONCURRENT_STREAMS = 1000",
                    "INITIAL_WINDOW_SIZE = 6291456",
                    "MAX_HEADER_LIST_SIZE = 262144"
                ]
            },
            {
                "frame_type": "WINDOW_UPDATE",
                "length": 4,
                "increment": 15663105
            },
            {
                "frame_type": "HEADERS",
                "stream_id": 1,
                "length": 451,
                "headers": [
                    ":method: GET",
                    ":authority: tls.peet.ws",
                    ":scheme: https",
                    ":path: /api/all",
                    "cache-control: max-age=0",
                    "sec-ch-ua: \\\".Not/A)Brand\\\";v=\\\"99\\\", \\\"Google Chrome\\\";v=\\\"103\\\", \\\"Chromium\\\";v=\\\"103\\",
                    "sec-ch-ua-mobile: ?0",
                    "sec-ch-ua-platform: \\\"Windows\\",
                    "upgrade-insecure-requests: 1",
                    "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
                    "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
                    "sec-fetch-site: none",
                    "sec-fetch-mode: navigate",
                    "sec-fetch-user: ?1",
                    "sec-fetch-dest: document",
                    "accept-encoding: gzip, deflate, br",
                    "accept-language: zh-CN,zh;q=0.9"
                ],
                "flags": [
                    "EndStream (0x1)",
                    "EndHeaders (0x4)",
                    "Priority (0x20)"
                ],
                "priority": {
                    "weight": 256,
                    "depends_on": 0,
                    "exclusive": 1
                }
            }
        ]
    }
}
DRGT23 commented 1 year ago

Do you alredy have a solution?

rottengeek commented 1 year ago

扩展中移除41

bogdanfinn commented 1 year ago

The issue here is the preSharedKey extension (41). we are currently working on a solution for that. but this has no eta yet.