net/http: document security considerations for serving internet traffic

[mmcloughlin]

The net/http/pprof package implicitly registers HTTP handlers through its init() function. I argue this implicit behavior is too subtle and may contribute to people inadvertently leaving such endpoints open. Some IPv4 scans reveal a non-trivial number of pprof endpoints exposed (http://mmcloughlin.com/posts/your-pprof-is-showing). Since Go 1.9 an exposed pprof endpoint leaks source code.

I would like to spark a discussion on the possibility of moving to an explicit handler registration model. It is not clear to me that this would qualify for the Security exemption of the Go 1 compatibility guarantee, but I thought it was worthy of mention. Perhaps a warning in documentation would be enough.

[dlsniper]

It only registers those handlers implicitly if you use the default http server instance. If you use a custom one, that doesn't happen. The problem is that people are using http.ListenAndServe without understanding its implications, not net/http/pprof imho. Furthermore, the default server usually needs tweaking when exposed to the Internet.

[mmcloughlin]

This is ultimately user error as you say. People are indeed using http.ListenAndServe and net/http/pprof without understanding its implications. I'm suggesting that this is an easy and understandable mistake, and we should strive to make the consequences as benign as possible.

If we were to address this by making backwards incompatible changes, I would suggest that replacing the init() function of net/http/pprof with a RegisterHandlers() (or similar) would be the most effective.

Arguably this is not serious enough to break compatibility. I think at least a warning in documentation is warranted.

[docmerlin]

I concur about a warning in documentation!

[mappu]

I understand the function names and file names are embedded into all binaries for panics and for reflection. I don't even particularly mind the pprof behaviour of attaching to the default mux.

But leaking source code is beyond the pale.

How does pprof get this information? Is it compiled into 1.9 binaries or loaded externally? If the former, is it only compiled into 1.9 binaries when pprof is used?

[dlsniper]

@mmcloughlin shouldn't you update your blog post if you think that the better solution is to not use the default ListenAndServe / http server and instead initialize your own?

As for Go documentations, PRs are welcomed by the team, so improving the usage example of net/http/pprof would be nice as well.

[4ad]

The source code is not embedded in the binary! go tool pprof has to have access to the source code in order to print the source code. The binaries only contain source code coordinates for symbols.

You ran all your tests on localhost, where pprof had access to the source of the server.

Of course, this does not mean the current functionality is not a footgun. I wish net/http/pprof would not work like this and I wish it didn't have a default mux, but it's too late to change any of that.

[mmcloughlin]

@4ad You're completely right, apologies for my mistake. Classic case of getting excited at a result and not doing the due diligence. Thanks for pointing this out. I'll update the post accordingly.

As you say there are still risks here people should be aware of.

@dlsniper I believe the Prevention section of the post covers this.

[rsc]

There are a number of packages that automatically register handlers under /debug/.... We chose that prefix specifically so that servers that serve direct internet traffic can put a handler in front of the http.DefaultServeMux to filter those out for untrusted clients.

@FiloSottile also has some tips about what to do as far as configuring other HTTP settings to avoid problems when serving direct internet traffic.

Probably a doc fix is correct, and probably http.ListenAndServe would be a good place, since (probably?) more people read the ListenAndServe docs than the pprof docs. Or maybe a link to a wiki page? Unclear.

[rsc]

It seems like perhaps the right next step is for someone to write a wiki page, and then the ListenAndServe docs can have a short link to the page. Putting lots of detail into the doc comment doesn't make as much sense as a wiki page we can update out-of-band.

[bradfitz]

Also to document somewhere: use of MaxBytesReader to limit the bytes read by Request.ParseForm, Request.FormValue, etc.

[tsuna]

Russ wrote:

@FiloSottile also has some tips about what to do as far as configuring other HTTP settings to avoid problems when serving direct internet traffic.

Just in case it's of any help to someone else wondering what those tips might be, a bunch of Googling revealed that Russ was most probably referring to this blog post: So you want to expose Go on the Internet, although that doesn't have anything to say about the things @bradfitz mentioned.

In any case, thanks @mmcloughlin for raising this issue, I was just about to push a change that would've exposed our debug endpoints to the Internets, the mistake is definitely very easy to make.

[bradfitz]

@FiloSottile, are you going to do this for Go 1.11?

[rsc]

Too late.

[mvdan]

As another doc suggestion, I think the godoc for net/http/pprof could explicitly say that the endpoints should, in general, not be made public.

[jdemeyer]

What I particularly like about http://mmcloughlin.com/posts/your-pprof-is-showing is that it gives examples how to do it right. If you decide to document the "don't do this" part, be sure to also document the "do that instead" part.

[liuzengh]

how about add go tag // +build !disable_init in net/http/pprof?

// +build !disable_init
func init() {
    http.HandleFunc("/debug/pprof/", Index)
    http.HandleFunc("/debug/pprof/cmdline", Cmdline)
    http.HandleFunc("/debug/pprof/profile", Profile)
    http.HandleFunc("/debug/pprof/symbol", Symbol)
    http.HandleFunc("/debug/pprof/trace", Trace)
}

[liuzengh]

we don't want net/http/pprof package implicitly registers HTTP handlers through its init() function because of security problem. Before go1.22, we use reflection to unregister http handlers that init at net/http/pprof. But the following code will failed at go1.22 because go 1.222 modify the implemention of http.ServeMux. Any better suggestion？

// unregisterHandlers delete router from http.DefaultServeMux.
// admin import net/http/pprof，Will automatically register pprof related routes on http.DefaultServeMux,
// avoid causing security problems.
// Can refer to：https://github.com/golang/go/issues/22085
func unregisterHandlers(patterns []string) error {
    // http.ServeMux struct：
    // type ServeMux struct {
    //  mu    sync.RWMutex
    //  m     map[string]muxEntry
    //  es    []muxEntry
    //  hosts bool
    // }

    // Need to import muxEntry in net/http pkg.
    type muxEntry struct {
        h       http.Handler
        pattern string
    }

    v := reflect.ValueOf(http.DefaultServeMux)

    // lock
    muField := v.Elem().FieldByName("mu")
    if !muField.IsValid() {
        return errors.New("http.DefaultServeMux does not have a field called `mu`")
    }
    muPointer := unsafe.Pointer(muField.UnsafeAddr())
    mu := (*sync.RWMutex)(muPointer)
    (*mu).Lock()
    defer (*mu).Unlock()

    // delete value of map.
    mField := v.Elem().FieldByName("m")
    if !mField.IsValid() {
        return errors.New("http.DefaultServeMux does not have a field called `m`")
    }
    mPointer := unsafe.Pointer(mField.UnsafeAddr())
    m := (*map[string]muxEntry)(mPointer)
    for _, pattern := range patterns {
        delete(*m, pattern)
    }

    // delete value of muxEntry slice.
    esField := v.Elem().FieldByName("es")
    if !esField.IsValid() {
        return errors.New("http.DefaultServeMux does not have a field called `es`")
    }
    esPointer := unsafe.Pointer(esField.UnsafeAddr())
    es := (*[]muxEntry)(esPointer)
    for _, pattern := range patterns {
        // removes muxEntry of same pattern.
        var j int
        for _, muxEntry := range *es {
            if muxEntry.pattern != pattern {
                (*es)[j] = muxEntry
                j++
            }
        }
        *es = (*es)[:j]
    }

    // modify hosts.
    hostsField := v.Elem().FieldByName("hosts")
    if !hostsField.IsValid() {
        return errors.New("http.DefaultServeMux does not have a field called `hosts`")
    }
    hostsPointer := unsafe.Pointer(hostsField.UnsafeAddr())
    hosts := (*bool)(hostsPointer)
    *hosts = false
    for _, v := range *m {
        if v.pattern[0] != '/' {
            *hosts = true
        }
    }

    return nil
}

[iTrooz]

It only registers those handlers implicitly if you use the default http server instance. If you use a custom one, that doesn't happen. The problem is that people are using http.ListenAndServe without understanding its implications, not net/http/pprof imho.

I think this is still a problem, because from the point of view of the user, they called http.ListenAndServe() and probably some http.HandleFunc(), but they never registered any debug endpoint against the http server, so it seems reasonable for them to think that they won't have debug endpoints present in their http server.

It also makes it harder to know whether debug endpoints are present or not for someone reading the code, and so it can be missed.

I don't think the fact this only happen with the default http server has anything to do with this, because the user still won't be expecting it.

For anyone wanting to stay away from net/http/pprof altogether to avoid confusion, google/gops can be a good alternative