Open Prinzhorn opened 2 years ago
I've realized how little trailers are used (aside from more specific uses such as in gRPC, which runs over HTTP/2). Also some of the tooling I'm using doesn't even support them, so that part is way less important than the chunked encoding. But I think the trailers are basically free to implement.
So, at a high level, I'm in favor of some combination of a) adding a new, more explicit and hopefully discoverable /chunked
endpoint b) making the Transfer-Encoding: chunked
behavior of the /stream
, /stream-bytes
, and /drip
[^1] endpoints more obvious/discoverable/explicit and c) adding the ability to specify trailers to any or all of the above.
Quick demo of /stream
using chunked transfer encoding:
$ curl --verbose --no-buffer --http1.1 https://httpbingo.org/stream/5 2>&1 | grep -e 'HTTP/1.1' -e 'transfer-encoding'
> GET /stream/5 HTTP/1.1
< HTTP/1.1 200 OK
< transfer-encoding: chunked
I suspect this will require a bit of fighting w/ the Golang stdlib net/http server's implementation, which automagically handles[^2] chunked transfer encoding in some(?) cases. We get that behavior implicitly in the endpoints above via our use of the http.Flusher
interface (docs, example usage in the handler for /stream
).
I've never dealt with trailers myself, so I have no idea what it would take to add that support to go-httpbin. As you point out the primary use case I've seen is for gRPC, but it should at least be possible.
[^1]: In poking around at this issue, I noticed that /drip
does not currently send back Transfer-Encoding: chunked
, at least as currently served by https://httpbingo.org/, though I expected it to. No idea if this is due to my host, or some difference in the way I'm using the Flusher interface in that handler (e.g. maybe writing a status code breaks something here). But this is kinda what I mean by having to fight w/ the stdlib to support the kind of functionality you have in mind!
[^2]: The automagical handling of Transfer-Encoding also makes it tough to write the kinds of test cases I'd like to write for these endpoints, as noted in this comment. Maybe there's a better way to do all of this, though, so any improvements here are welcome!
Thanks for looking into this. All the things you've listed sound reasonable. I didn't know that some endpoints already support chunked.
I'll probably get to the integration tests within the next three month. From what I can tell the exiting endpoints might already get me almost 100% of the way.
I'm planning on using go-httpbin for integration tests of an HTTP intercepting proxy. Thanks for maintaining it!
I will come up with some more "exotic" feature requests and have to see how easy it is to add those myself.
I'd love to see support for chunked encoding with trailers.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Trailer https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing
Here's a rough idea:
/chunked/:n
Generates a chunked response withn
random chunks of variable size, aTransfer-Encoding: chunked
header and an optionalServer-Timing
trailerParameters:
minsize=1
: Min number of random bytesmaxsize=1024
: Max number of random bytestrailer=1
: Include theTrailer: Server-Timing
header andServer-Timing
trailerDoes that sound like something you would accept? Do you see any problems? I don't really care about
Server-Timing
directly, but I need any trailer.Foo
would do but I thought we might as well use something that's actually used IRL and has some browser support.To support
max-body-size
we could just check the worst casemaxsize * n
before doing anything.