No way to force a chunk.

[dpc]

I am working on a project using chunking response streaming, that requires flushing current data at any arbitrary moment. One problem that I have hit is that there's no way to force a chunk edge at a given place in a returned response in goproxy.

I am returning a Response where Body is a PipeReader, and I have a coroutine streaming the data into coresponding PipeWriter. By default net/http will just use 4K buffer and send chunks about every 4K, and only w.(http.Flusher).Flush() (where w is ResponseWriter) allows to force a chunk edge. There's no way to do it through Pipe. In ServeHTTP goproxy will just io.Copy(w, resp.Body) which will just pass the data from PipeReader to ResponseWriter.

I think the same problem was mentioned by other people too: https://groups.google.com/forum/#!msg/goproxy-dev/hFEoLkBwhdM/CvnYGQao8XcJ

[elazarl]

Hmmm.... do you have an idea how to solve it, while still providing a http.Response?

On Wed, Apr 27, 2016 at 3:26 AM, Dawid Ciężarkiewicz < notifications@github.com> wrote:

I am working on a project using chunking response streaming, that requires flushing current data at any arbitrary moment. One problem that I have hit is that there's no way to force a chunk edge at a given place in a returned response in goproxy.

I am returning a Response where Body is a PipeReader, and I have a coroutine streaming the data into coresponding PipeWriter. By default net/http will just use 4K buffer and send chunks about every 4K, and only w.(http.Flusher).Flush() (where w is ResponseWriter) allows to force a chunk edge. There's no way to do it through Pipe. In ServeHTTP goproxy will just io.Copy(w, resp.Body) which will just pass the data from PipeReader to ResponseWriter.

I think the same problem was mentioned by other people too: https://groups.google.com/forum/#!msg/goproxy-dev/hFEoLkBwhdM/CvnYGQao8XcJ

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/elazarl/goproxy/issues/159

[dpc]

I was thinking about custom modification of io.Pipe that would have a flush_need flag: Writerimplementing aFlusherinterface andPipeReaderimplementing a customFlushReceiverinterface, and a customio.Copythat could detect if the reader is implementsFlushReceiverand callFlushon theResponseWriter`.

Somewhat complicated but I think it would work.

[dpc]

Also, there's no way to use https://golang.org/pkg/net/http/#Hijacker . The above solution could fix that too, by implementing this interface for that custom modification of io.Pipe.

But I start to believe that making hooks return something other than http.Response would be better. Then that something can implement Flush and Hijack, and no performance and compilcations of pipe are necessary.