x/net/httpproxy: useProxy rejects localhost/loopback addresses unnecessarily

[ianw]

The useProxy function automatically returns false if it decides the target host is localhost or a loopback address @ https://cs.opensource.google/go/x/net/+/master:http/httpproxy/proxy.go;drc=c6ed85c7a12db1bd15e993fd3ae4700b2e9f2c84

if host == "localhost" {
    return false
}
ip := net.ParseIP(host)
if ip != nil {
    if ip.IsLoopback() {
        return false
    }
}

It appears to be just about the only proxy implementation that does this. A good gitlab blog has done a comparison at https://about.gitlab.com/blog/2021/01/27/we-need-to-talk-no-proxy/

There seem to be many valid cases for wanting communications to localhost also proxied. For example, I have setup a docker registry I would like to test listening at localhost:9000 and wish to usemitmproxy on port 8080 to dump the traffic between a client and this testing registry, to analyse the API calls the client is making.

If you do the obvious thing of setting up docker to use http_proxy=http://localhost:8080 the traffic is never proxied to localhost:9000. I'm not sure anyone really expects this; for example even the docker proxy information page https://docs.docker.com/network/proxy/ examples are showing setting 127.0.0.0/8 in the noProxy (maybe it works differently on Windows, where you do require this to not proxy localhost? That seems to make the difference just even more confusing. Searching shows plenty of other examples of people not assuming that localhost/127* are being excluded automatically).

Perhaps there are older comments, I wasn't sure how to track back further than https://cs.opensource.google/go/x/net/+/c7086645de248775cbf2373cf5ca4d2fa664b8c1

Is there any reason why this loopback avoidance shouldn't just be removed in favour of just obeying what is in no_proxy, to bring this in line with most other implementations?

[cagedmantis]

cc @neild @ianlancetaylor

[p53]

i stumbled on this problem, i wanted to use ProxyFunc in my project to not duplicate PROXY/NO_PROXY functionality and standardise but i am unable to write local tests for this because of localhost restriction

[gopherbot]

Change https://go.dev/cl/577535 mentions this issue: net/httpproxy: remove loopback check in useProxy func Fixes golang/go#51416

[p53]

also how would one use e.g. sidecar in pod which would use this function wanting to listen on localhost...