nytimes / gziphandler

Go middleware to gzip HTTP responses
https://godoc.org/github.com/NYTimes/gziphandler
Apache License 2.0
868 stars 129 forks source link

io.Copy error when minSize > 0 #39

Closed frankban closed 7 years ago

frankban commented 7 years ago

In the cases when a Write is performed and minSize has not been reached yet (in essence when the check at [1] is false) the number of written bytes as returned by the method is 0. This means that when using io.Copy a ErrShortWrite is returned and the copy fails (see [2]). Perhaps this behavior should be documented, or Write should pretend len(b) bytes have been written in that case (as they will be eventually)?

[1] https://github.com/NYTimes/gziphandler/blob/master/gzip.go#L107 [2] https://golang.org/src/io/io.go#L400

adammck commented 7 years ago

Oops. This was raised in #32, but I forgot to follow up. The number returned by Write doesn't imply any kind of durability, simply that the bytes were "written". (bytes.Buffer, which we use under the hood, works in the same way.) So we should fix this. Thanks for the report.

flimzy commented 7 years ago

Perhaps this behavior should be documented,

I think this behavior is a bug. If 0 is returned, the caller should expect that re-calling write() with the same data would be idempotent; but it's not in this case. So I agree with @adammck that "Write doesn't imply any kind of durability", and this bug should be fixed, rather than documented.

In the meantime, I've vendored an old version of this library until this bug can be fixed.

mikegleasonjr commented 7 years ago

I just hit this as well.

In my case I am returning a multipart form data response:

parts := multipart.NewWriter(w)             // w here is a http.ResponseWriter wrapped by a gziphandler
parts.CreateFormFile("asset", "test.html")  // errors with 'short write'

So I guess I just can set a minSize of 0 for now?

mikegleasonjr commented 7 years ago

Thanks for the quick fix, will test that out

mikegleasonjr commented 7 years ago

It raised another error:

When I have a buffer of a known length and try to write to the gzip handler, I get more bytes written than I originally sent:

b := f.buf[f.off:]
l := len(b)           // 104870
n, err := w.Write(b)  // w is my gziphandler (writer)
// n is 105079

Want me to open another issue?

EDIT: opened #44