About Transparent Proxy

[lls275]

First of all, I am very grateful to the AdguardTeam team for giving so many excellent open source projects, from which I have learned a lot. Recently, I have been fascinated by the gomitmproxy project. The code of this project is very elegant, but I want to enter gomitmproxy through iptables traffic on Openwrt, but Encountered many setbacks.

So, I would like to ask you to help me, now I will describe the problem in detail:

First of all, there is no problem with HTTP proxying, it works fine. But I wanted to use a transparent proxy, so I replaced &net.TCPAddr with net.ResolveTCPAddr:

//addr := &net.TCPAddr{IP: listenIP, Port: options.ListenPort}
addr, err := net.ResolveTCPAddr("tcp", ":12345")

config.ProxyConfig = gomitmproxy.Config{
        ListenAddr: addr,

        MITMConfig:     mitmConfig,
        MITMExceptions: []string{"example.com"},
    }

sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to 10.20.1.1:12345
sudo iptables -t nat -A PREROUTING -p tcp --dport 443 -j DNAT --to 10.20.1.1:12345

It seems that the code does not report an error, but when it is actually run, a huge problem occurs. First of all, if you access the content of port 80, there seems to be no big problem, and you can access it normally:

2022/04/07 08:12:41 9756#56 [debug] id=100004-1: handle request GET http://local.local.wooc.org/cart
2022/04/07 08:12:41 9756#56 [debug] urlfilter: id=100004-1: saving session
2022/04/07 08:12:41 9756#56 [debug] id=100004-1: request was overridden by: http://local.wooc.org/cart
2022/04/07 08:12:41 9756#56 [debug] id=100004-1: response was overridden by: 200 OK
2022/04/07 08:12:41 9756#56 [debug] id=100004: waiting for request
2022/04/07 08:12:42 9756#51 [debug] id=100005: accepted connection from 10.20.1.92:54267

But when accessing the link to 443, it cannot be accessed, and a bunch of garbled characters are prompted:

2022/04/07 08:12:42 9756#35 [debug] id=100005: failed to read request: malformed HTTP request "\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03**\xa41\x92\x963K\x84\xb5\xe6f{>W\xb9\xcbVL\xe5\xc1[\xa2I\xeb#\x96xH\u0094\xe7 \xd2=\xa27\x1a\xb83\xa7?[\x83s(\xcfy\xeeʞ\xcfJ\x97`\xeb\x03\xb2;v\xfaJO\xf65\x006\xca\xca\x13\x01\x13\x02\x13\x03\xc0,\xc0+̩\xc00\xc0/̨\xc0$\xc0#\xc0"
2022/04/07 08:12:42 9756#61 [debug] id=100005: closing connection due to: malformed HTTP request "\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03**\xa41\x92\x963K\x84\xb5\xe6f{>W\xb9\xcbVL\xe5\xc1[\xa2I\xeb#\x96xH\u0094\xe7 \xd2=\xa27\x1a\xb83\xa7?[\x83s(\xcfy\xeeʞ\xcfJ\x97`\xeb\x03\xb2;v\xfaJO\xf65\x006\xca\xca\x13\x01\x13\x02\x13\x03\xc0,\xc0+̩\xc00\xc0/̨\xc0$\xc0#\xc0"
2022/04/07 08:12:42 9756#51 [debug] id=100006: accepted connection from 10.20.1.92:54268

It should be a CONNECT request, but the above is not.

I have been troubled by this problem for a long time. I hope the official can help me when I see my post, and I will continue to support you with more and better products.

[lls275]

I'm desperately asking for help right now, I can only hope for now

[lls275]

I extracted a simple example of the core http proxy of the gomitmproxy, which is printed normally in http

func main() {
    listenAddr := &net.TCPAddr{
        IP:   net.IPv4(0, 0, 0, 0),
        Port: 8080,
    }
    l, err := net.ListenTCP("tcp", listenAddr)
    if err != nil {
    }
    log.Printf("start listening to %s", l.Addr())

    conn, err := l.Accept()
    log.Printf("remote addr is %s", conn.RemoteAddr())
    log.Printf("local addr is %s", conn.LocalAddr())

    localRW := bufio.NewReadWriter(bufio.NewReader(conn), bufio.NewWriter(conn))

    var req *http.Request
    reqc := make(chan *http.Request, 1)
    errc := make(chan error, 1)

    go func() {
        r, err := http.ReadRequest(localRW.Reader)
        if err != nil {
            errc <- err
            return
        }
        reqc <- r
    }()

    req = <-reqc

    log.Println(req)
}

It can be obtained normally, but if the proxy is transparent, how should this code be changed?

2022/04/07 15:52:09 start listening to [::]:8080
2022/04/07 15:52:11 remote addr is 192.168.1.23:59812
2022/04/07 15:52:11 local addr is 192.168.1.23:8080
2022/04/07 15:52:11 &{CONNECT //www.google.com:443 HTTP/1.1 1 1 map[Proxy-Connection:[keep-alive] User-Agent:[Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4774.0 Safari/537.36]] {} <nil> 0 [] false www.google.com:443 map[] map[] <nil> map[]  www.google.com:443 <nil> <nil> <nil> <nil>}

[lls275]

If the official can give a simple example of modification, it would be great, thank you very much

[ameshkov]

Well, the problem is that currently it simply does not support the transparent proxy approach, it expects to be used as a normal proxy. The code requires substantial changes to support this.

[lls275]

Well, the problem is that currently it simply does not support the transparent proxy approach, it expects to be used as a normal proxy. The code requires substantial changes to support this.

If you add a transparent proxy, roughly what you need to do? Is it first to initiate a CONNECT request to gomitmproxy through TCP for iptable traffic, when encountering HTTPS traffic? It seems that if a third-party transparent proxy (support upstream is HTTP https://github.com/xiqingping/transproxy) is connected between iptable and gomitmproxy, it can be used, but I want to integrate the transparent proxy function into gomitmproxy (this way I don’t need to open another program), but due to my limited personal ability, I can’t implement it yet. Can the author provide some ideas?