Danny-Dasilva / CycleTLS

Spoof TLS/JA3 fingerprints in GO and Javascript
GNU General Public License v3.0
916 stars 172 forks source link

Add a way to not canonicalize headers #278

Closed aj3423 closed 4 months ago

aj3423 commented 11 months ago

Description

Currently headers are canonicalized, for example:

func main() {
    client := cycletls.Init()
    client.Do(
        `http://...`,
        cycletls.Options{
            Headers: map[string]string{
                `Abc`: ``,
            },
        },
        `GET`,
    )
}

The ABC is canonicalized to Abc, need a way to not canonicalize it, cause some site is sensitive.

Issue Type

Feature Request

Operating System

Linux

Node Version

None

Golang Version

Other

Relevant Log Output

No response

Danny-Dasilva commented 11 months ago

I can include this in the next release. It would be helpful for me if you had a test url.

as far as what is standard casing shouldn't matter for http2 requests

From RFC 7540, section 8.1.2:

Just as in HTTP/1.x, header field names are strings of ASCII characters that are compared in a case-insensitive fashion. However, header field names MUST be converted to lowercase prior to their encoding in HTTP/2. A request or response containing uppercase header field names MUST be treated as malformed (Section 8.1.2.6).

aj3423 commented 11 months ago

Hi, here're two python servers for testing:

For http:

from http.server import BaseHTTPRequestHandler, HTTPServer

class RequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.print_headers()

    def do_POST(self):
        self.print_headers()

    def print_headers(self):
        print("")
        for header_name, header_value in self.headers.items():
            print(f"{header_name}: {header_value}")

with HTTPServer(('', 80), RequestHandler) as httpd:
    httpd.serve_forever()

And for https:

from http.server import BaseHTTPRequestHandler, HTTPServer
import ssl

class RequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.print_headers()

    def do_POST(self):
        self.print_headers()

    def print_headers(self):
        print()
        for header_name, header_value in self.headers.items():
            print(f"{header_name}: {header_value}")

        self.send_response(200)
        self.end_headers()

ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)

ssl_context.load_cert_chain(certfile="localhost.crt", keyfile="localhost.key")

with HTTPServer(('', 443), RequestHandler) as httpd:
    httpd.socket = ssl_context.wrap_socket(httpd.socket)
    httpd.serve_forever()

Key files are generated with:

openssl req -x509 -out localhost.crt -keyout localhost.key   -newkey rsa:2048 -nodes -sha256   -subj '/CN=localhost' -extensions EXT -config <( \
   printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")

Verified with: curl -k http://localhost -H 'AAA: bbb' or curl -k https://localhost -H 'AAA: bbb'

The server shows:

Host: localhost User-Agent: curl/8.2.1 Accept: / AAA: bbb <------------------------- 127.0.0.1 - - [28/Nov/2023 02:45:06] "GET / HTTP/1.1" 200 -

sussyGaymer commented 9 months ago

@Danny-Dasilva Any updates?