can't get chunked response to work; ProtocolVersion is wrong? (HttpListenerMode.EmbedIO only)

[eli-darkly]

Describe the bug I'm trying to write a response with chunked encoding. Instead, what gets sent is a non-chunked response. This happens only if I use HttpListenerMode.EmbedIO instead of HttpListenerMode.Microsoft, and it looks like the problem is related to HttpListenerResponse.ProtocolVersion being set incorrectly.

To Reproduce

Run this code:

            var options = new WebServerOptions()
                    .WithUrlPrefix($"http://*:10000")
                    .WithMode(HttpListenerMode.EmbedIO);
            var server = new WebServer(options);
            server.OnAny("/", async ctx =>
            {
                ctx.Response.SendChunked = true;
                await ctx.SendStringAsync("chunk1,", "text/plain", Encoding.UTF8);
                await ctx.SendStringAsync("chunk2", "text/plain", Encoding.UTF8);
            });
            _ = server.RunAsync();
            while (true)
            {
                Thread.Sleep(1000);
            }

(Here's an archive of the buildable project: TestApp.zip)

While that's running, make an HTTP request from the command line: curl -i --raw http://localhost:10000/

Expected behavior The response should look like this:

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked

7
chunk1,
6
chunk2
0

Actual behavior Instead, I get a non-chunked response that includes only the first chunk. Also note that the protocol version is 1.0, which doesn't support chunked encoding. That's happening even though the request protocol is 1.1 (adding -v to the curl command shows this).

HTTP/1.0 200 OK
Content-Type: text/plain; charset=utf-8

chunk1,

Environment I'm using .NET Core 2.1 on MacOS 10.15.5.

Additional context Debugging into the code showed that ctx.Response.ProtocolVersion is 1.0. However, the version in the request (ctx.Request.ProtocolVersion) is 1.1, which is correct.

I suspect that the problem might be that HttpListenerResponse.ProtocolVersion is set in the constructor to be the same as the request's protocol version— but, the constructor is being called immediately after constructing the request, and the request has not yet parsed the protocol version yet; that doesn't happen until SetRequestLine is called a bit later.

[eli-darkly]

It seems to work fine if I use HttpListenerMode.Microsoft. I'm not entirely clear on what the advantages are of using one versus the other.

[rdeago]

Hello @eli-darkly, thanks for using EmbedIO!

<sarcasm> I'd like to award you an EmbedIO T-shirt or something for finding the 1000^th HttpListener bug, but I lost count months ago. </sarcasm>

No small thanks to your thorough research, this one looks pretty easy to fix: since the response's protocol version must necessarily be the same as the request's, HttpListenerResponse.ProtocolVersion's getter may just return the request's ProtocolVersion. This means adding a reference to the request in HttpListenerResponse, but

I just had a look at .NET 5's HttpListener sources. Their managed HttpResponse has both:

• an own ProtocolVersion property, which is initialized to HttpVersion.Version11 and then never touched again;
• a reference to the request object, whose ProtocolVersion is checked when necessary.

It was a fun thing to discover, so thanks for that too.

To me, it looks like HttpListenerResponse.ProtocolVersion may as well not exist on its own but instead just return the request's ProtocolVersion. This way we can leave the rest of the code untouched.

511 should fix this issue. If you don't want to wait for the release of EmbedIO v3.5.0, you can test by building from source: no special tools required, just Visual Studio or the .NET SDK.

[eli-darkly]

Great - glad that it turned out to have a straightforward solution, and thanks for the quick fix (this isn't a blocker for me though, since using HttpListenerMode.Microsoft seems to be fine for my purposes).

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[markmeeus]

i just stumbled over this issue. Is there any release date in sight for 3.5?

[rdeago]

Hello @markmeeus, sorry for bringing bad news... There's no plan for a 3.5 version any more.

EmbedIO v4.0, however, will solve this using a "stub" response and not creating a HttpListenerResponse at all until request processing is finished. This will also solve a bunch of HttpListener-related problems.

I have to ask everybody to be patient: EmbedIO is on my priority list, I'm still using it in new projects, but I'm sorta overbooked at the time, working full time on code that will benefit both EmbedIO and other projects... besides, I also have to pay bills like everybody. I had to take the hard decision to stop working on version 3.5 and go for 4.0 directly, in order to solve some "structural" problems once and for all, two of them being (case in point) our tight binding to HttpListener and our reliance on an old and buggy implementation. I'm not saying version 4 will free us from HttpListener completely, but it will pave the way for a "pluggable" HTTP layer, so when HttpListener is deprecated or even dropped from the .NET runtime we can decide whether to "unplug" it and use something else, fork it and maintain it separately, or maybe some other option that will surface in the future.