bogdanfinn / tls-client

net/http.Client like HTTP Client with options to select specific client TLS Fingerprints to use for requests.
BSD 4-Clause "Original" or "Old" License
668 stars 133 forks source link

[Bug]: Case issue in response protocol header #119

Open jwwsjlm opened 5 days ago

jwwsjlm commented 5 days ago

TLS client version

v1.3.12

System information

windows10,64

Issue description

微信截图_20240627004657 微信截图_20240627004733 Case issue in response protocol header

As shown in the figure, I used net/http to create an httpServe, and the response protocol header was set to x-custom-header. Note that it is lowercase. When tested with other tools, the responses were all lowercase. However, when using tls_client.NewHttpClient-client.Do(req) to send a request, taking resp.Header, it will become uppercase. I know this is a standard. But is there any way to get the original format?

Steps to reproduce / Code Sample

`func main() {

client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger())

req, err := fhttp.NewRequest(fhttp.MethodGet, "http://127.0.0.1:55881/ping", nil)

resp, err := client.Do(req)
if err != nil {
    log.Println(err)
    return
}

defer resp.Body.Close()

for key, values := range resp.Header {
    fmt.Printf("Key: %s, Values: %v\n", key, values)
}
log.Println(fmt.Sprintf("status code: %d", resp.StatusCode))

readBytes, err := io.ReadAll(resp.Body)
if err != nil {
    log.Println(err)
    return
}

log.Println(string(readBytes))

}`

jwwsjlm commented 5 days ago

QQ图片20240627005557 Use other tools. httpserver returns lowercase.

combo23 commented 5 days ago

it's possible when you force HTTP1, this can be set in client options

jwwsjlm commented 5 days ago

it's possible when you force HTTP1, this can be set in client options

Is there any way to get the original response protocol header?

combo23 commented 5 days ago

what do you mean

jwwsjlm commented 5 days ago

what do you mean

https://github.com/luminati-io/luminati-proxy

Hmm. I mainly want to use it to develop a middleware or something like that. Other languages ​​can use golang to send requests through a proxy. But there is a problem that the net\http package will correct the case of the protocol header. For other clients, they need to be handled separately, which is very troublesome. Something like this

jwwsjlm commented 5 days ago

what do you mean

However, when the middleware forwards data, the server returns lowercase letters, and the middleware sends it to the client in uppercase letters, causing the client to not recognize it.

So I hope to get the raw bytes or unprocessed response headers

jwwsjlm commented 5 days ago

what do you mean

This is indeed not in compliance with the specification. However, as a middleware, it can perfectly copy what the client requests and what the server responds to. In the middle, tls-client is used to perform tls fingerprint randomization.