net/http: using io.Copy to copy a file to http.ResponseWriter does not use sendfile

[terrywh]

Go version

go version go1.22.6 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/root/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/root/go'
GOPRIVATE=''
GOPROXY='https://*********'
GOROOT='/root/server/go'
GOSUMDB='******************'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/root/server/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.6'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/data/tmp/go-build1438424244=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Implement a Handler for http.Server and try to streaming a file:

func (handler Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  file, _ := os.Open("file")
  io.Copy(w, file)
}

What did you see happen?

io.Copy not using sendfile() when copy from *os.File to http.ResponseWriter:

1. io.Copy first try to use WriterTo from *os.File, but *http.response does not implement syscall.Conn interface;
2. fallback to genericWriteTo in WriteTo of *os.File;
3. file got wrapped in os.fileWithoutWriteTo (file.go:269)
4. another call to io.Copy try to use *http.response.ReadFrom() which calls sendfile()
5. which trys to use TCPConn.ReadFrom()
6. src is not a *os.File but a os.fileWithoutWriteTo wrapper. (sendfile_linux_go:20)
7. fallback to genericReadFrom() which will do all the copying stuff (tcpsock_posix.go:54)

What did you expect to see?

calling sendfile() in either *os.File.WriteTo Or *http.response.ReadFrom;

[prattmic]

cc @neild

[neild]

See #67074.

[gabyhelp]

Related Issues and Documentation

• net: sendFile can't handle os.fileWithoutWriteTo #66988 (closed)
• net/http: slow io.Copy(http.ResponseWriter, *os.File) with Content-Length #41513
• net/http: permit splice from socket to ResponseWriter #40888 (closed)
• net: interleaving sendfile and read calls are very slow #45256
• net.sendFile does not use sendfile() on linux in all the cases in which it is available #7005 (closed)
• net/http: request body transfer optimization issues (ReadFrom/sendfile) #30377 (closed)
• io: Copy() unnecessarily allocates memory when src is a file and is empty #53658 (closed)
• affected/package: io.copy() ; did not achieve the expected result. #50090 (closed)
• net/http: ServeFile 2X slower in Go 1.21 #61530 (closed)
• x/net/http2: response Write on HEAD fails with "http2: request body closed due to handler exiting" #66812

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}