hysteria2 masquerade options updated to let modern websites load correctly

doggy commented 5 days ago

Removed r.Out.Host = r.In.Host to resolve an issue where the proxy was incorrectly using the client's host header instead of the target's host. This ensures the proxy request aligns with the intended masquerade behavior.

API Document

https://pkg.go.dev/net/http/httputil#ProxyRequest.SetURL

SetURL rewrites the outbound Host header to match the target's host.

Currently the code line r.Out.Host = r.In.Host is used for preserve the inbound request's Host header which is sing-box server's host name in our situation.

Sample Code

A go proxy sample here for occurring this bug:

go version go version go1.23.3 darwin/arm64

cat proxy.go


package main

import (
    "log"
    "net/http"
    "net/http/httputil"
    "net/url"
)

func main() {
    masqueradeURL, _ := url.Parse("https://bing.com")
    masqueradeHandler := &httputil.ReverseProxy{
        Rewrite: func(r *httputil.ProxyRequest) {
            r.SetURL(masqueradeURL)
            r.Out.Host = r.In.Host
        },
        ErrorHandler: func(w http.ResponseWriter, r *http.Request, err error) {
            log.Printf("Proxy error: %v", err)
            w.WriteHeader(http.StatusBadGateway)
        },
    }

    http.Handle("/", masqueradeHandler)
    log.Println("Starting reverse proxy...")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

The bad result:

➜  ~ curl -v http://localhost:8080
* Host localhost:8080 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8080...
* Connected to localhost (::1) port 8080
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 400 Bad Request
< Date: Wed, 20 Nov 2024 16:46:16 GMT
< X-Msedge-Ref: 0WRI+ZwAAAADVvLX68eHdSrZSJ6s7q3ALQlkzRURHRTA0MTAARWRnZQ==
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< 
* Connection #0 to host localhost left intact
<h2>Our services aren't available right now</h2><p>We're working to restore all services as soon as possible. Please check back soon.</p>

After removed r.Out.Host = r.In.Host, it seems works properly.

➜  ~ curl -v http://localhost:8080
* Host localhost:8080 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8080...
* Connected to localhost (::1) port 8080
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 301 Moved Permanently
< Accept-Ch: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
< Cache-Control: private
< Content-Security-Policy: script-src https: 'strict-dynamic' 'report-sample' 'wasm-unsafe-eval' 'nonce-YdgXJAbl+5fxQkfvbJK4CuovCM+Hg+zVVIFsZe5oN8w='; base-uri 'self';
< Content-Type: text/html; charset=utf-8
< Date: Wed, 20 Nov 2024 16:46:43 GMT
< Location: https://www.bing.com:443/?toWww=1&redig=37A991C2208A4FEDBBF54E5BE47699F0
< Set-Cookie: MUID=3989612CA23B6FF82A2A7411A3776E11; domain=bing.com; expires=Mon, 15-Dec-2025 16:46:43 GMT; path=/; secure; SameSite=None
< Set-Cookie: MUIDB=3989612CA23B6FF82A2A7411A3776E11; expires=Mon, 15-Dec-2025 16:46:43 GMT; path=/; HttpOnly
< Set-Cookie: _EDGE_S=F=1&SID=1F12B189E4FB631B15E3A4B4E5B76278; domain=bing.com; path=/; HttpOnly
< Set-Cookie: _EDGE_V=1; domain=bing.com; expires=Mon, 15-Dec-2025 16:46:43 GMT; path=/; HttpOnly
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< Useragentreductionoptout: A7kgTC5xdZ2WIVGZEfb1hUoNuvjzOZX3VIV/BA6C18kQOOF50Q0D3oWoAm49k3BQImkujKILc7JmPysWk3CSjwUAAACMeyJvcmlnaW4iOiJodHRwczovL3d3dy5iaW5nLmNvbTo0NDMiLCJmZWF0dXJlIjoiU2VuZEZ1bGxVc2VyQWdlbnRBZnRlclJlZHVjdGlvbiIsImV4cGlyeSI6MTY4NDg4NjM5OSwiaXNTdWJkb21haW4iOnRydWUsImlzVGhpcmRQYXJ0eSI6dHJ1ZX0=
< Vary: Accept-Encoding
< X-Cache: CONFIG_NOCACHE
< X-Eventid: 673e12738d1d4715835655e99b215794
< X-Msedge-Ref: Ref A: 14012417CC364045A4F102B297BC103A Ref B: BY3EDGE0408 Ref C: 2024-11-20T16:46:43Z
< Transfer-Encoding: chunked
< 
<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="https://www.bing.com:443/?toWww=1&amp;redig=37A991C2208A4FEDBBF54E5BE47699F0">here</a>.</h2>
</body></html>
* Connection #0 to host localhost left intact

nekohasekai commented 3 days ago

Maybe we need to add a new option for this, since this is the default behavior of hysteria2: https://github.com/apernet/hysteria/blob/15e31d48a09af0835773bd5c62cedbb3fc0e351d/app/cmd/server.go#L815

doggy commented 2 days ago

Maybe we need to add a new option for this, since this is the default behavior of hysteria2: https://github.com/apernet/hysteria/blob/15e31d48a09af0835773bd5c62cedbb3fc0e351d/app/cmd/server.go#L815

Thank you for mentioning this code. It’s weird that the default settings for HTTPS masquerade do not work with the most popular websites like bing, yelp..

This PR updates the masquerade option style to include support for rewriteHost and making it more similar to Hysteria2. It also retains support for the previous Sing-Box style for better understanding among existing users.

doggy commented 2 days ago

Docs and unit test updated also

SagerNet / sing-box

hysteria2 masquerade options updated to let modern websites load correctly #2297

API Document

Sample Code