SSL works in Ubuntu Docker container on desktop, but fails in Docker on Ubuntu Server

[YudrB]

Description

started with a configuration using nginx proxy serving multiple web sites. 1) first problem occurred when trying to deploy a website with .net Identity using the external MicrosoftAccounts provider.

• no problems making this work on local host but it fails when deployed to the web server.
• Initially I thought his was a proxy forwarding problem
• but it turns out SSL is required between the webserver and proxy.
• Is this a bug? Seems unintuitive and I found no reference to it in any of the documentation. The generated error is also misleading. It complains that redirect_uri doesn't match.
• The only reason they don't match is because the underlying transport is not SSL and something in the Identity handshake notices.
• Ideally I'd like to be able to use Identity without SSL since my proxy server guarantees SSL externally.

2) Second problem occurred when trying to use SSL with the docker container.

• SSL in a docker container on the dev machine, with the dev certificate works fine.
• Production SSL certificates from letsencrypt work fine on the dev machine also.
• But they failed with docker on the Ubuntu server. (even without the proxy).
• Documentation was not clear on how this should be configured. Server just refuses connection.
• After much trial and error, I discovered that if you specify the cert.pem and privkey.pem in the appsettings.json then it works. This should be documented in the walkthroughs.

"Kestrel": {
  "EndPoints": {
    "Http": {
      "Url": "http://0.0.0.0"
    },
    "HttpsFromPem": {
      "Url": "https://0.0.0.0:443",
      "Certificate": {
        "Path": "/etc/letsencrypt/live/apointlesswasteoftime.com/fullchain.pem",
        "KeyPath": "/etc/letsencrypt/live/apointlesswasteoftime.com/privkey.pem"
      }
    }
  }
}

Reproduction Steps

• create a razor web project.
• Get production certificates from letsencrypt
• setup locally with production certificates in a docker container. Note that it works.
• Delploy the docker container to a ubuntu server. Note that it doesn't work unless you use the specific configuration setting in the appsettings.json. I could not get this to work by configuring the certs in the programs.cs

Expected behavior

expect that the behavior between the ubuntu docker container on the dev machine should be identical the behavior on the ubuntu server. This is a bug. I suspect it is due to some ASP.NET interactions with the networking stack?

Actual behavior

Works fine in Ubuntu container on desktop. I can even point the nginx proxy at the desktop container and it works fine. Just didn't work on server without specifying the certs in the appsettings.json.

Regression?

don't know.

Known Workarounds

specifying the certs in the appsettings.json rather than in programs.cs

Configuration

.Net 8 & 7 both appear to repro this. Latest Ubuntu and Docker all systems are x64

Other information

No response

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones See info in area-owners.md if you want to be subscribed.

Issue Details

---

### Description started with a configuration using nginx proxy serving multiple web sites. 1) first problem occurred when trying to deploy a website with .net Identity using the external MicrosoftAccounts provider. - no problems making this work on local host but it fails when deployed to the web server. - Initially I thought his was a proxy forwarding problem - but it turns out SSL is required between the webserver and proxy. - Is this a bug? Seems unintuitive and I found no reference to it in any of the documentation. The generated error is also misleading. It complains that redirect_uri doesn't match. - The only reason they don't match is because the underlying transport is not SSL and something in the Identity handshake notices. - Ideally I'd like to be able to use Identity without SSL since my proxy server guarantees SSL externally. 2) Second problem occurred when trying to use SSL with the docker container. - SSL in a docker container on the dev machine, with the dev certificate works fine. - Production SSL certificates from letsencrypt work fine on the dev machine also. - But they failed with docker on the Ubuntu server. (even without the proxy). - Documentation was not clear on how this should be configured. Server just refuses connection. - After much trial and error, I discovered that if you specify the cert.pem and privkey.pem in the appsettings.json then it works. This should be documented in the walkthroughs. "Kestrel": { "EndPoints": { "Http": { "Url": "http://0.0.0.0" }, "HttpsFromPem": { "Url": "https://0.0.0.0:443", "Certificate": { "Path": "/etc/letsencrypt/live/apointlesswasteoftime.com/fullchain.pem", "KeyPath": "/etc/letsencrypt/live/apointlesswasteoftime.com/privkey.pem" } } } } ### Reproduction Steps - create a razor web project. - Get production certificates from letsencrypt - setup locally with production certificates in a docker container. Note that it works. - Delploy the docker container to a ubuntu server. Note that it doesn't work unless you use the specific configuration setting in the appsettings.json. I could not get this to work by configuring the certs in the programs.cs ### Expected behavior expect that the behavior between the ubuntu docker container on the dev machine should be identical the behavior on the ubuntu server. This is a bug. I suspect it is due to some ASP.NET interactions with the networking stack? ### Actual behavior Works fine in Ubuntu container on desktop. I can even point the nginx proxy at the desktop container and it works fine. Just didn't work on server without specifying the certs in the appsettings.json. ### Regression? don't know. ### Known Workarounds specifying the certs in the appsettings.json rather than in programs.cs ### Configuration .Net 8 & 7 both appear to repro this. Latest Ubuntu and Docker all systems are x64 ### Other information _No response_

Author:	YudrB
Assignees:	-
Labels:	`area-System.Net.Security`
Milestone:	-

[wfurt]

How is that related to .NET runtime @YudrB? It seems like problem either with your container setup, nginx or Kestrel.

[ghost]

This issue has been marked needs-author-action and may be missing some important information.

[YudrB]

Might be Kerstel related. But since I was following a .net walkthrough on Identity when I hit the issue, I thought it best to start here. At the very least some documentation updates would help others not hit this problem. But the Identity issue appears to be a framework issue since it requires SSL, even though it isn't really needed. That makes me suspect that the SSL issue may also be .Net related since some of the failures happen while trying to process the identity login. Pr at least appear to. I've had very little problem setting up reverse proxy to other containers. .Net was the first time I hit any significant issues. Thanks for looking and hope reporting this helps someone else.

[rzikm]

Are you following any tutorial/walkthrough from .NET foundation? Can you post a link to the specific tutorial?

but it turns out SSL is required between the webserver and proxy.

That is entirely dependent on the configuration of proxy and .NET identity

Production SSL certificates from letsencrypt work fine on the dev machine also.

Define "work". Lots of things influence whether SSL "works". E.g. as part of validating the server certificate, the hostname of the certificate is validated against the hostname the client is connecting to. I assume your local dev machine has different hostname than the server you are deploying to?

Delploy the docker container to a ubuntu server. Note that it doesn't work unless you use the specific configuration setting in the appsettings.json. I could not get this to work by configuring the certs in the programs.cs

There should be no difference whether the cert is loaded manually or specified in appsettings.json.

What would very much help is a minimal repro (e.g. zipped source code directory with everything that is necessary) with instructions how to run it. Without it all the description is too vague and it is not possible for us to help.

[ghost]

This issue has been marked needs-author-action and may be missing some important information.

[YudrB]

Its been a while since I looked at this. I don't believe I found a walkthrough that explained how to configure a server in production. The docs appear to only discuss the dev environment. However for SSL there where two walkthroughs that seemed relevant: https://learn.microsoft.com/en-us/aspnet/web-api/overview/security/working-with-ssl-in-web-api and https://tutorials.eu/how-to-configure-ssl-in-asp-net/#:~:text=cs%20file%20of%20your%20ASP,UseHttpsRedirection()%3B

For the Nginx, I used: https://learn.microsoft.com/en-us/troubleshoot/developer/webapps/aspnetcore/practice-troubleshoot-linux/2-2-install-nginx-configure-it-reverse-proxy

and for Identity I used: https://learn.microsoft.com/en-us/aspnet/core/security/authentication/identity?view=aspnetcore-8.0&tabs=visual-studio

somewhere in all those documents it should explain that SSL is required on the internal servers for identity to work. Even though SSL is already being handled at the nginx proxy. The error generated is not helpful in tracking down the problem. The root of this problem is that on the local dev machine the identity framework doesn't require ssl, or will work with the self signed certs. I'm guessing Localhost is a special case for Identity. However on a production server, ssl appears to be manditory, but the error message doesn't tell you that. In my config the nginx proxy is configured to handle the ssl and then forwards the headers to the server. This is how I would expect most people would initially setup the systems.

for the SSL configuration on the server. I ended up with 3 different configs. 1) VS with IIS on the dev machine, that seems to work out of the box with the self signed certs. 2) For local docker testing on dev machine I had to create self signed certs manually and put them in a pfx that I included with the project. "Docker": { "commandName": "Docker", "launchBrowser": true, "launchUrl": "{Scheme}://{ServiceHost}:{ServicePort}", "environmentVariables": { "ASPNETCORE_ENVIRONMENT": "Development", "ASPNETCORE_KestrelCertificatesDefault__Password": "password", "ASPNETCORE_KestrelCertificatesDefault__Path": "/app/aspnetapp.pfx", "ASPNETCORE_HTTPS_PORTS": "443" }, 3) for the production system I have the below, since I couldn't get this to work in code. Maybe it was just me? But it would have been nice to have the local docker scenario use the same code as the server. "AllowedHosts": "*", "Kestrel": { "EndPoints": { "Http": { "Url": "http://0.0.0.0" }, "HttpsFromPem": { "Url": "https://0.0.0.0:443", "Certificate": { "Path": "/etc/letsencrypt/live/logicmedia.com-0001/fullchain.pem", "KeyPath": "/etc/letsencrypt/live/logicmedia.com-0001/privkey.pem" } } } }

Hope that helps. I reported this mainly because I've found a few places now, where the documentation works great while in the dev environment, but moving it into product can be a challenge if you don't understand what the underlying code is expecting. Hopefully it saves someone else a couple of days of wondering why isn't this working?

Thx.

[wfurt]

I moved this to aspnetcore in case there is opportunity for better doc.