HTTP/2 + SSL to HTTP proxying

[HarelM]

Hi, First and foremost, thanks a lot for this great library! I've been using this in production for a while now and it is working great! We have recently did a migration from a Windows server to a linux server. The migration was also out of IIS and into kestrel as the server in the front. I'm guessing that this change allowed browsers to start communicating though the HTTP/2 protocol. My question is how does this package support this. The question comes from a problem we just observer. The problem is as follows:

1. The browser (or even a curl client) calls our kestrel server to check if it supports HTTP/2, it does.
2. It then sends a request to the server (over SSL and HTTP/2)
3. The server proxies the request over HTTP to another "hidden" server.
4. This "hidden" server return a HTTP response with headers and everything (HTTP style)
5. The response is sent as is to the client
6. The client, expecting a HTTP/2 response with no headers is telling us the response is invalid (which is correct in terms of spec).

Chrome and FireFox are OK with this, but curl and safari are not and we see issues with our iOS users.

Are you guys familiar with this? Is this a know limitation? Do I need to configure something to make this work? My proxy code can be found here: https://github.com/IsraelHikingMap/Site/blob/508259dd1c16466f9b00711581ab48da8ecbd431/IsraelHiking.Web/Startup.cs#L174

[twitchax]

Hello!

Glad to hear that this library is working out well for you. I think the best method would be to add to the options, and strip the headers. Sort of like this.

proxies.Map(proxyEntry.Key,
    proxy => proxy.UseHttp(
      (_, args) => { /* your normal endpoint computer logic */ },
      options => options.WithAfterReceive((context, response) => response.Headers.Clear())
));

So, the first thing to note here is that UseHttp has another optional parameter called builderAction which allows you to pass in an "options builder", which is basically just an instance of the default options. Then, you can mutate those options.

Basically, the WithAfterReceive handler allows you to edit the response message from the upstream server. The only caveat here is that you might need to do something different depending on the request type. In that case, you would inspect the context to see whether it is an HTTP/1.X request (don't strip headers) or HTTP/2 request (strip headers). I'm not sure exactly what logic you want to have, but you should have (1) all the information you need about the "context" of the request that came into the proxy server (that's in context), and (2) the ability to edit the message that came from the upstream (that's in response) before it goes back down to the client.

Let me know if this works. If not, we'll figure out something. :)

[HarelM]

Thanks for the super quick response! I'll give it a go later tonight. Just out of curiosity, shouldn't this be a part of this library and act as default case when returning a http2 response?

[twitchax]

Yes, and that gives me a better idea. ASP.NET might do the magic for us automatically. It's not that HTTP/2 doesn't have headers: it's that HTTP/2 doesn't have plaintext headers.

Let me see if I can automatically detect the version override the response.

[HarelM]

Thanks! I can confirm that clearing the headers solves the issue, but I would prefer a better solution. Let me know if you want me to test a pre-release NuGet version or something of that sort...

[twitchax]

Hi @HarelM, yeah, I agree.

The problem is that stripping the headers is not technically the right solution. HTTP/2 allows for headers, but they are part of the body, I think. I need to do more research to discover the exact correct response.

[HarelM]

I tried looking at the code in this repo tonight. The following code copies the response message into the http context (which should be HTTP/2, shouldn't it?). https://github.com/twitchax/AspNetCore.Proxy/blob/274bd4c0cbc715029edb2593dc05c7290055b076/src/Core/Extensions/Http.cs#L122 Interestingly I don't see the version being copied (which is expected), so I'm not sure why the headers are not "converted" properly as the response is updated in the original http context... I looked at this SO question: https://stackoverflow.com/questions/53764083/use-http-2-with-httpclient-in-net I'm probably still missing something... The docs on this are so limited I'm having a hard time finding anything related to this in google...

[mpashkov]

Hello. I am going to describe my problem. Maybe, you can help me to resolve it the best way.

I have the web-site which I reach throught the proxy. During the execuption of the request I get the exceptoin with the message: "The SSL connection could not be established, see inner exception. (the message of the inner exceptoin is The remote certificate is invalid according to the validation procedure: RemoteCertificateNameMismatch)".

When proxy tries to reach this site it gets redirect to another site with https-schema. However, http-header with name "Host" does not change its value. As a result, RemoteCertificateNameMismatch happend.

I remove this header and it works fine.

builderAction: builder =>
                            builder
                                .WithBeforeSend(
                                    beforeSend: (_, message) =>
                                    {
                                        message.Headers.Remove("Host");
                                        return Task.CompletedTask;
                                    })

If you want, I may send the url which exception occured in private message. I am not sure that I may publish this here.

It is great pleasure to get your opinion about this situation. Thank you.

[twitchax]

@mpashkov, this sounds like something that is unique to the server. You may need to set the Host header to the Common Name of the server's certificate.

[mpashkov]

@twitchax , please, may you investigate the link if you have a little bit time?

[HarelM]

Just a note, we have moved from using this library to use Nginx and this solved our issue related to http2. I guess it has to do with the implementation of Nginx vs .net core. Thanks for all the hard work invested in this project! It served us well. 🙏