search

dotnet / AspNetCore.Docs

Documentation for ASP.NET Core
https://docs.microsoft.com/aspnet/core
Creative Commons Attribution 4.0 International
12.6k stars 25.29k forks source link

How to setup the dev certificate when using Docker in development #6199

Closed Rick-Anderson closed 5 years ago

Rick-Anderson commented 6 years ago

related #3310 Javier is contact: This needs to go in Enforce HTTPS in an ASP.NET Core The first time you run dotnet after installing the SDK you get this message Successfully installed the ASP.NET Core HTTPS Development Certificate. To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only). For establishing trust on other platforms please refer to the platform specific documentation. For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.

Copied from #3310 We also need to cover how to setup the dev certificate when using Docker in development:

Rick-Anderson commented 6 years ago

Subject: RE: Docker for ASP.NET Core 2.1 Preview 2

. Provided that your docker file and docker compose look like the ones that VS generates when you add VS support, there are a couple of steps you need to take to enable it manually. In essence, you need to get the HTTPS development certificate from your machine into the container image as a pfx file in a special location on disk that we recognize and provide the password through user.

• Modify the dockerfile to expose the port 443 with • EXPOSE 443 • Modify the docker-compose override file to map ports, volumes and environment variables as follows: environment:

per Javier: For setting up a non development certificate on kestrel the instructions are the same as for the development case (obviously you need to provide the certificate) and you need to set the path to the certificate in a configuration key. "Kestrel:Certificates:Default:Path"

csharpfritz commented 6 years ago

Is there an update for this now they we are in RTM for 2.1 and preparing for 2.1.1 release?

robinmanuelthiel commented 6 years ago

Can we please avoid things like "Create an application on Visual Studio using the MVC template." in the documentation for a cross-platform framework like .NET Core, that also runs on Linux and Mac devices?

RaccoonDev commented 6 years ago

Hi.

The method works fine if I run the application in development environment. Is there are way to specify what certificate and port should kestrel use in production mode?

dradovic commented 6 years ago

When using a pfx file generated by dotnet dev-certs on my local Windows machine, I ran into a

error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure

So I switched to simply calling dotnet dev-certs https --export-path ./localhost.pfx -p $LOCALHOST_CERTIFICATE_PWD as part of building the Docker image and then referencing that in Startup.cs using KestrelServerOptions:

listenOptions.UseHttps("../../localhost.pfx", Environment.GetEnvironmentVariable("LOCALHOST_CERTIFICATE_PWD"));

kdcllc commented 6 years ago

I had an issue with Docker-Compose.yml and Visual Studio.Net 15.7.5. The resolution was adding the entry for secret,json:

{
    "Kestrel":{
        "Certificates":{
            "Default":{
                "Path":     "/root/.aspnet/https/<AppName>>.pfx",
                "Password": "<<Your-Password>>"
            }
        }
    }
}
se-augustus commented 6 years ago

Do these same rules apply to MacOS users? How about those of us using Kitematic? I'm wondering if I can extrapolate the notes above to fit into the areas on/in the attached screenshots.

screen shot 2018-08-24 at 8 08 40 pm screen shot 2018-08-24 at 8 08 56 pm
chrisckc commented 6 years ago

In essence, you need to get the HTTPS development certificate from your machine into the container image as a pfx file in a special location on disk that we recognize and provide the password through user.

I have tried this in Docker and unless I specify the certificate location and password in the Startup configuration it crashes with: System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.

I followed the guide provided by @Rick-Anderson which seems to imply that UseStartup() now checks for the existence of the cert at root/.aspnet/https/<>.pfx or checks the UserSecrets for Kestrel:Certificates:Development:Path and Kestrel:Certificates:Development:Password

The UserSecrets is working and configured correctly as i am using it to get the path and password, it works using this code:

                .UseKestrel(options =>
                {
                    var configuration = (IConfiguration)options.ApplicationServices.GetService(typeof(IConfiguration));
                    var httpsPort = configuration.GetValue("ASPNETCORE_HTTPS_PORT", 443);
                    var certPassword = configuration.GetValue<string>("Kestrel:Certificates:Development:Password");
                    var certPath = configuration.GetValue<string>("Kestrel:Certificates:Development:Path");

                    options.Listen(IPAddress.Any, httpsPort, listenOptions =>
                    {
                        listenOptions.UseHttps(certPath, certPassword);
                    });
                });
chrisckc commented 6 years ago

I have found that if the path to the cert is specified in user secrets or an env var, it breaks.

I was able to get it to work using the default startup by removing the path which i added when trying a bunch of other things: dotnet user-secrets remove "Kestrel:Certificates:Development:Path"

I can break it again by re-adding the path: dotnet user-secrets set "Kestrel:Certificates:Development:Path" "/root/.aspnet/https/<<AppName>>.pfx"

The path is correct and the cert file exists as demonstrated in the code above, but for some reason it doesn't like having the path specified even if it is correct. The guide above does only say to add the password, but there should be no reason that adding the correct path, or even any valid path where the cert exists could not be made to work.

This only applies to Development environment, in Production environment it works as per the docs using the 'Default' path: dotnet user-secrets set "Kestrel:Certificates:Default:Path" "/root/.aspnet/https/<<AppName>>.pfx"

chrisckc commented 6 years ago

I have published a repo which uses this technique which can be used to reproduce this issue: https://github.com/chrisckc/DotNetCoreAureliaSPA

Rick-Anderson commented 6 years ago

@danroth27 what's the priority on this? We might need some help - at least for the first draft. @chrisckc could you provide a draft?

chrisckc commented 6 years ago

@Rick-Anderson I should be revisiting that repo soon to build something out so will have a dig around to see what is causing the issue i posted about.

chrisckc commented 6 years ago

Ok, i traced back the error message:

System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.

which occurs when "Kestrel:Certificates:Development:Path" is specified in the configuration from UserSecrets or env vars despite the path being correct and the file existing.

I cloned the KestrelHttpServer repo and traced the message back to the Load() method inside the KestrelConfigurationLoader class (line 216).

The Load() method calls LoadDefaultCert(ConfigurationReader) LoadDefaultCert checks if the "Kestrel:Certificates:Default" config entry exists, if so then the cert is loaded using the LoadCertificate(CertificateConfig certInfo, string endpointName) method which just builds the path using Path.Combine(env.ContentRootPath, certInfo.Path) , which if certInfo.Path is an absolute path just returns certInfo.Path due to the way Path.Combine works.

However if the "Kestrel:Certificates:Default" config entry does not exist is call FindDeveloperCertificateFile(configReader, logger) which is where things get a little odd.

The following particularly ugly piece of code is used to locate the development certificate:

               if (configReader.Certificates.TryGetValue("Development", out var certificateConfig) &&
                    certificateConfig.Path == null &&
                    certificateConfig.Password != null &&
                    TryGetCertificatePath(out certificatePath) &&
                    File.Exists(certificatePath))
                {
                    var certificate = new X509Certificate2(certificatePath, certificateConfig.Password);
                    return IsDevelopmentCertificate(certificate) ? certificate : null;
                }
                else if (!File.Exists(certificatePath))
                {
                    logger.FailedToLocateDevelopmentCertificateFile(certificatePath);
                }

The TryGetCertificatePath(out certificatePath) method looks for a cert named $"{appName}.pfx" in any of the following paths to cover both Mac OSX and Windows: $APPDATA/ASP.NET/https, $APPDATA/.aspnet/https, $HOME/ASP.NET/https, $HOME/.aspnet/https

That piece of code is completely broken if it's intention is to either use the certificateConfig.Path or go and find the certificate file from a default location and naming convention, which would be the sensible thing to do.

The result of that 'if' statement is that if a certificate path is supplied in the config it is never used.

It also will report a misleading error because a missing password would result in a log error which says that the certificate could not be located and also an exception inside the Load() method stating that that certificate could not be found.

Experiencing misleading errors has always been an issue when working with Microsoft software in the past, was hoping that the situation has been improving.

I also noticed that the IsDevelopmentCertificate method checks is the certificate.Subject is equal to "CN=localhost", which may not always be the case for every developers development environment. For example if you are developing using docker running on another machine or a VM etc.

The code can be simplified, i don't really see the need to use the 2 separate configuration structures: "Kestrel:Certificates:Default" and "Kestrel:Certificates:Development" and check the subject of the cert. Just one config with a check to try and find the cert from the default location and name if it is not supplied in ":Path"?

chrisckc commented 6 years ago

Looking at the referenced issue on line 332 of KestrelConfigurationLoader.cs

https://github.com/aspnet/Hosting/issues/1294

It looks like the broken code was added as part of that, which seemed to start off with good intentions. Maybe the behaviour is intentional, if so i would like to know the reasoning.

@Rick-Anderson Any documentation written to cover this setup with its current behaviour would look rather odd.

danroth27 commented 6 years ago

@javiercn @Tratcher

javiercn commented 6 years ago

That piece of code is completely broken if it's intention is to either use the certificateConfig.Path or go and find the certificate file from a default location and naming convention,

It's not meant to be used with a path, the development certificate key is only there to support tooling scenarios. If you want to specify a path and a password, simply use the Default key.

I also noticed that the IsDevelopmentCertificate method checks is the certificate.Subject is equal to "CN=localhost", which may not always be the case for every developers development environment.

It's only meant to support localhost scenarios (including docker localhost). Other scenarios will have security implications we would have to worry about.

It also will report a misleading error because a missing password would result in a log error which says that the certificate could not be located and also an exception inside the Load() method stating that that certificate could not be found.

There's no way AFAIK to check the validity of the password on the PFX.

PrefabPanda commented 6 years ago

I've been trying to follow this example and I'm tearing my hair out!

I get these exceptions: AuthenticationException: The remote certificate is invalid according to the validation procedure. HttpRequestException: The SSL connection could not be established, see inner exception.

I have two containers, one talks to the other by HTTPS and by use of an alias configured in the compose file. The container being talked to doesn't have a public IP on purpose (Gateway Microservice architecture).

I have generated a certificate and am applying it to the service being called.

Any ideas?

PrefabPanda commented 6 years ago

Turns out I needed this: https://blog.roushtech.net/2016/12/20/ignoring-ssl-certificate-errors-net-core-httpclient/

and yes this is in my development environment.

simonz130 commented 6 years ago

Hey folks, this also affects scenarios when you try things out in Azure Shell or other similar shells. When you do:

dotnet new mvc

then 

dotnet run

in first instance of Azure Shell, then open a new instance to test it out with 

curl https://localhost:5001

you get:

curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

Ideally, I shouldn't need to use -k or -i.

javiercn commented 6 years ago

I don't know very well the scenario for azure shell or other shells, but you should be able to disable https when you create the app through dotnet new (I think -no-https?).

The development HTTPS certificate is designed for development on the development machine. For "cloud native development" scenarios you need to either disable https or create a certificate and find a way to plumb it through your specific deployment scenario. In this case using -k should be fine.

Also please keep in mind that it is not recommended to install the SDK on a production machine (as it contains things and setup used for development) and that the recommended approach is to either use the runtime image (as we do in our multi image docker builds) or at least disable the certificate generation on the local box by setting the DOTNET_GENERATE_ASPNET_CERTIFICATE environment variable to false.

simonz130 commented 6 years ago

You can read about Azure Shell here. It's used to try things out from documentation, learning or even quick&dirty development sometimes. Biggest advantage of this is that you don't need anything installed on your local machine and it's easily disposable. This is not considered production machine - it's a docker container that has storage mounted to it.

killnine commented 5 years ago

Maybe I am doing this incorrectly but perhaps this line needs to change:

Export the HTTPS certificate into a PFX file using the dev-certs global tool to ${HOME}/.aspnet/https/<>.pfx using a password of your choice (dotnet dev-certs https -ep ${HOME}/.aspnet/https/ -p <>)

to

Export the HTTPS certificate into a PFX file using the dev-certs global tool to ${HOME}/.aspnet/https/<>.pfx using a password of your choice (dotnet dev-certs https -ep ${HOME}/.aspnet/https/yourCertName.pfx -p <>)

When I did it without a filename specified I got the error:

There was an error exporting HTTPS developer certificate to a file.

EDIT: It is insane how frustrating setting this all up is.

disophisis commented 5 years ago

This may not be the best thread for this question, but:

We are using docker-compose to setup services that are dependent on one another, so we have external links from one of those services to several others. We have dev-certs setup for all of them, and they all work independently, but the service with the external links does not trust the dev-certs, so container to container traffic over https is not working. the dotnet dev-certs tool apparently does not have a --trust option in Linux. Is there some documentation on trusting those certs for container to container traffic in linux?

javiercn commented 5 years ago

@disophisis There is no general way to trust certificates across linux distributions, it varies from distro to distro. The certificate generated by dotnet dev-certs is meant to be used for more basic scenarios where you are containerizing an app into a single container.

For the topology that you have, I imagine your machine to machine traffic uses service names instead of localhost, so my recommendation is to generate individual https certificates for those services through openssl or powershell, configure kestrel to use the appropriate certificate on each service and do whatever it takes to trust other certificates across the services. In this scenario it might make sense for you to create a CA certificate for development and generate TLS certificates with the CA certificate as the root, so that you only need to trust the CA certificate across services.

Hope this helps

shanselman commented 5 years ago

Help me understand the issue here now that VS 15.9.1 does it automatically? Can we document what VS is doing automatically on Windows (with a Linux Container) so it's easier everywhere? image

rynnova commented 5 years ago

I'm currently running into a similar problem, in that I want to have a base docker-compose.yml file for common configuration, a docker-compose.override.yml file for development and a docker-compose.prod.yml file for production, using external secrets and configs.

My current issue is that, since I'm using a reverse proxy configuration, according to the note here, I shouldn't need https from Kestrel. However, the code for the application doesn't have any mention of https, secure services or anything else, and yet after building a Dockerfile for this application and running nginx, aspnetcore and sql-server on Linux containers, I still get this damnable Unhandled Exception: System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found. error.

Why would ASP.NET still require HTTPS in this instance? Would WsFederation authentication have anything to do with it?

Tratcher commented 5 years ago

No in-app components will request https for you, it has to come from config. Have you tried logging the config as observed by the app? https://github.com/aspnet/KestrelHttpServer/blob/5c1fcd664d39db8fe5c8e38052a3cc29f90322f6/samples/SampleApp/Startup.cs#L178-L185

rynnova commented 5 years ago

I have not, I will try this after the holidays and get back to you then.

rynnova commented 5 years ago

Okay, I put that section in my Program.cs:

using System;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Server.Kestrel.Core;
using Microsoft.Extensions.Configuration;

namespace Vue2Spa {
    public class Program {
        public static void Main(string[] args) {
            CreateWebHostBuilder(args).Build().Run();
        }

        public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .UseKestrel(LogDevelopmentConfiguration)
                .UseStartup<Startup>();

        private static void LogDevelopmentConfiguration(
            WebHostBuilderContext context,
            KestrelServerOptions options) {
            if (context.HostingEnvironment.IsDevelopment())
                ShowConfig(context.Configuration);
        }

        private static void ShowConfig(IConfiguration configuration) {
            foreach (var pair in configuration.GetChildren()) {
                Console.WriteLine($"{pair.Path} - {pair.Value}");
                ShowConfig(pair);
            }
        }
    }
}

However, nothing gets logged and it still fails with the certificate error.

app_1_70b2dc094d0a | : Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[0]
app_1_70b2dc094d0a |       User profile is available. Using '/root/.aspnet/DataProtection-Keys' as key repository; keys will not be encrypted at rest.
app_1_70b2dc094d0a | info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
app_1_70b2dc094d0a |       Creating key {8b718e81-eb89-4b52-ae29-918542806596} with creation date 2018-11-26 11:28:54Z, activation date 2018-11-26 11:28:54Z, and expiration date 2019-02-24 11:28:54Z.
app_1_70b2dc094d0a | warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
app_1_70b2dc094d0a |       No XML encryptor configured. Key {8b718e81-eb89-4b52-ae29-918542806596} may be persisted to storage in unencrypted form.
app_1_70b2dc094d0a | info: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
app_1_70b2dc094d0a |       Writing data to file '/root/.aspnet/DataProtection-Keys/key-8b718e81-eb89-4b52-ae29-918542806596.xml'.
app_1_70b2dc094d0a | crit: Microsoft.AspNetCore.Server.Kestrel[0]
app_1_70b2dc094d0a |       Unable to start Kestrel.
app_1_70b2dc094d0a | System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.
app_1_70b2dc094d0a | To generate a developer certificate run 'dotnet dev-certs https'. To trust the certificate (Windows and macOS only) run 'dotnet dev-certs https --trust'.
app_1_70b2dc094d0a | For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions, Action`1 configureOptions)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IServerAddressesFeature addresses, KestrelServerOptions serverOptions, ILogger logger, Func`2 createBinding)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
app_1_70b2dc094d0a | 2018-11-26 11:29:15,365 [1] FATAL Microsoft.AspNetCore.Server.Kestrel - Unable to start Kestrel.
app_1_70b2dc094d0a |
app_1_70b2dc094d0a | Unhandled Exception: System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.
app_1_70b2dc094d0a | To generate a developer certificate run 'dotnet dev-certs https'. To trust the certificate (Windows and macOS only) run 'dotnet dev-certs https --trust'.
app_1_70b2dc094d0a | For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions, Action`1 configureOptions)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IServerAddressesFeature addresses, KestrelServerOptions serverOptions, ILogger logger, Func`2 createBinding)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Hosting.Internal.WebHost.StartAsync(CancellationToken cancellationToken)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String shutdownMessage)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token)
app_1_70b2dc094d0a |    at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host)
app_1_70b2dc094d0a |    at Vue2Spa.Program.Main(String[] args) in /source/Vue2Spa/Program.cs:line 14
app_app_1_70b2dc094d0a exited with code 139

EDIT: If it helps, I have made a gist containing the basic setup of the application.

javiercn commented 5 years ago

@ryakstis You are telling Kestrel to listen on https on your dockerfile https://gist.github.com/ryakstis/4ea1c9883b6cec4c770fd5dfb1e100ae#file-dockerfile-L39

rynnova commented 5 years ago

@javiercn haha, wow, I feel dumb now. That should have been obvious, but I never thought to look at environment variables when thinking about configuration. Thank you!

If it helps, I did manage to use the microsoft/dotnet:2.1-sdk-alpine image to resolve the dev-cert issue temporarily, since that automatically includes a dev cert. Knowing I can change the URL format helps make that no longer necessary.

javiercn commented 5 years ago

@ryakstis No problem. Its easy to miss details like that.

datvm commented 5 years ago

Sorry but things in this thread is quite confusing for me as I am quite new to Docker. Please help me with these:

FROM microsoft/dotnet:2.1-aspnetcore-runtime AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

It would be great if someone could make a details steps on these.

(I am using VS version 15.9.3 and .NET Core 2.1.1)

rynnova commented 5 years ago

@datvm

bilalmalik777 commented 5 years ago

I am trying to run my localhost to https in docker container but i am getting the following error (given in very next double quotes) and it is working perfectly over the http. I also attache the image of error "curl: (35) schannel: failed to receive handshake, SSL/TLS connection failed" I am using the following technology Asp.net core 2.0, Docker version 18.09.0,

Docker compose is given bellow ` version: '3.4'

services:

tt.core.web: image: ${DOCKER_REGISTRY-}ttcoreweb build: context: . dockerfile: TT.Core.Web/Dockerfile environment:

networks: app-network: driver: bridge

` fbcapture

javiercn commented 5 years ago

The environment needs to be development for the dev certificate to work

bilalmalik777 commented 5 years ago

Yòu mean it should be development in place of devdocker? Devdocker is my development environment

javiercn commented 5 years ago

@bilalMlaik The Development certifícate only gets loaded for the Development environment. If you want to use a different key for your development environment then you need to load the certificate manually providing the path to the certificate and the password in config.

bilalmalik777 commented 5 years ago

@javiercn i loaded it manually in usekesteral method program.cs file by giving path of .pfx and userscret key in appsetting.json. but i am not giving the path of certificate.cert. the .pfx and key is getting move into container in the specified path

kdcllc commented 5 years ago

@bilalMlaik this documentatiom might be helpful Hosting ASP.NET Core Images with Docker over HTTPS.

I ran into situation where if I am using dotnet dev cert. I have to add manually the following to the apps secret:

  "Kestrel:Certificates:Development:Password": "43f5d164-3813-4dba-8e34-6337096f2df0"

The id must match the project secret entry:

    <UserSecretsId>8c5d84b3-27c0-4964-a32b-91dd9b1c01d3</UserSecretsId>
chrisckc commented 5 years ago

@shanselman With regard to the original issue, yes it works out-of-the-box without any additional config when using Visual Studio but is fragile due to the issue i raised earlier in this thread. Users of VSCode who look to set this up manually may run into issues.

I have tested the out-of-the-box experience on both Windows and MacOS:

Windows Using Visual Studio Community 2017 v15.9.3 with dotnet v2.2 SDK and "Docker For Windows" (Docker Desktop Community Version 2.0.0.0-win81 (29211)) File > New > Project > Visual C# > .Net Core > ASP.NET Core Web Application and then check "Docker support"

vs-new-project-docker

Run the app in Debug (F5) works without error after allowing the certificate trust prompts (look out for it in the background) , the app then runs from the local docker container on HTTPS using self signed localhost certificate.

The certificate is volume mapped into the container from the local filesystem, here is how Visual Studio builds and runs the container (for the benefit of VSCode users):

docker build -f "C:\Users\username\source\repos\DotNetCoreMvcTest\DotNetCoreMvcTest\Dockerfile" -t dotnetcoremvctest:dev --target base  --label "com.microsoft.created-by=visual-studio" "C:\Users\username\source\repos\DotNetCoreMvcTest"
docker run -dt -v "C:\Users\username\vsdbg\vs2017u5:/remote_debugger:rw" -v "C:\Users\username\source\repos\DotNetCoreMvcTest\DotNetCoreMvcTest:/app" -v "C:\Users\username\AppData\Roaming\ASP.NET\Https:/root/.aspnet/https:ro" -v "C:\Users\username\AppData\Roaming\Microsoft\UserSecrets:/root/.microsoft/usersecrets:ro" -v "C:\Users\username\.nuget\packages\:/root/.nuget/fallbackpackages2" -v "C:\Program Files\dotnet\sdk\NuGetFallbackFolder:/root/.nuget/fallbackpackages" -e "DOTNET_USE_POLLING_FILE_WATCHER=1" -e "ASPNETCORE_ENVIRONMENT=Development" -e "ASPNETCORE_URLS=https://+:443;http://+:80" -e "ASPNETCORE_HTTPS_PORT=44348" -e "NUGET_PACKAGES=/root/.nuget/fallbackpackages2" -e "NUGET_FALLBACK_PACKAGES=/root/.nuget/fallbackpackages;/root/.nuget/fallbackpackages2" -p 50286:80 -p 44348:443 --entrypoint tail dotnetcoremvctest:dev -f /dev/null

Notice the 2 volume mappings:

"C:\Users\username\AppData\Roaming\ASP.NET\Https:/root/.aspnet/https:ro"
"C:\Users\username\AppData\Roaming\Microsoft\UserSecrets:/root/.microsoft/usersecrets:ro"

The self signed Certificate generated for the project is here: "C:\Users\username\AppData\Roaming\ASP.NET\Https\DotNetCoreMvcTest.pfx" and is made available in the container at "/root/.aspnet/https/DotNetCoreMvcTest.pfx"

The UserSecrets JSON file is here: "C:\Users\username\AppData\Roaming\Microsoft\UserSecrets{guid}\secrets.json" and is made available in the container at "/root/.microsoft/usersecrets/{guid}/secrets.json"

Contents of secrets.json (the certificate password is a generated guid, unrelated to the UserSecretsId)

{
  "Kestrel:Certificates:Development:Password": "{guid}"
}

The cert path is not required as dotnet looks for "/root/.aspnet/https/{AppName}.pfx"

Ok, now in reference to the issues that i raised: https://github.com/aspnet/Docs/issues/6199#issuecomment-418194220 let's break it:

Open up command prompt inside the project directory and run the command: dotnet user-secrets set "Kestrel:Certificates:Development:Path" "/root/.aspnet/https/DotNetCoreMvcTest.pfx"

contents of secrets.json are now this:

{
  "Kestrel:Certificates:Development:Password": "22c3f8ae-4857-4ab7-a8e6-570ed636b85f",
  "Kestrel:Certificates:Development:Path": "/root/.aspnet/https/DotNetCoreMvcTest.pfx"
}

Now run the app and Boom!

System.InvalidOperationException: 'Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.

This should not happen as the path is actually correct, the excuse that a path is not meant to be used in development is lame, the option is there so it should work, a developer may wish to use a different dev cert, rename the cert or move the cert.

It's easy to fix in the source, i even went to the trouble of finding out the cause: https://github.com/aspnet/Docs/issues/6199#issuecomment-427834083

Setting this up does need to be documented, but it should be clearly stated in the docs that a path can't be used with "Kestrel:Certificates:Development" as the dev cert handling in KestrelConfigurationLoader is broken.

MacOS Visual Studio for Mac Community v7.7.1 (build 15) with dotnet v2.2 SDK and "Docker For Mac" (Docker Desktop Community Version 2.0.0.0-mac81 (29211)) File > New Solution > .NET Core > App > ASP.NET Core Web App (MVC) The new project workflow is a little different and there is no option for docker support upon project creation in Visual Studio for Mac, it's just a mildly updated rebrand of Xamarin Studio.

Once the project is created, right click on the project > Add > Add Docker Support This creates a Docker file and a docker-compose file whereas VS on Windows only created a Dockerfile when Docker support was selected. (to get docker compose support on Windows, right click on project > Add > "Container Orchestrator support" and select docker compose.)

Running the app reveals that it does not work, failing with the following:

Error: Creating network "dockercompose........_default" with the default driver The path /usr/local/share/dotnet/sdk/NuGetFallbackFolder\r\nis not shared from OS X and is not known to Docker

To fix that, add this path to the Docker File sharing settings "/usr/local/share/dotnet/sdk/NuGetFallbackFolder" This requirement is not documented anywhere as far as i know.

The app can be broken in the same way by attempting to specify the dev cert path.

Summary The easiest way to get working Docker / Docker Compose support when using VSCode is to create the project using Visual Studio / Visual Studio for Mac.

chrisckc commented 5 years ago

For anyone who wishes to know how VS run the project when docker compose is used (Container Orchestrator support) , here are the build and run commands extracted from VS on Windows.

BTW. make sure that you run 'docker stop' against the container that was generated before adding "Container Orchestrator support" otherwise the build will fail with:

Bind for 0.0.0.0:{port} failed: port is already allocated

Build:

docker-compose  -f "C:\Users\username\source\repos\DotNetCoreMvcTest\docker-compose.yml" -f "C:\Users\username\source\repos\DotNetCoreMvcTest\docker-compose.override.yml" -f "C:\Users\username\source\repos\DotNetCoreMvcTest\obj\Docker\docker-compose.vs.debug.g.yml" -p dockercompose9785789912942731059 --no-ansi build

Run:

docker-compose  -f "C:\Users\username\source\repos\DotNetCoreMvcTest\docker-compose.yml" -f "C:\Users\username\source\repos\DotNetCoreMvcTest\docker-compose.override.yml" -f "C:\Users\username\source\repos\DotNetCoreMvcTest\obj\Docker\docker-compose.vs.debug.g.yml" -p dockercompose9785789912942731059 --no-ansi up -d --no-build --force-recreate --remove-orphans

As can be seen there is a "docker-compose.override.yml" file referenced, which can be created if required, and also a secret docker-compose file "docker-compose.vs.debug.g.yml" that is added to the mix by VS, this file is transient so i can't easily grab the contents but i can get the resulting docker compose config that is generated as its included in the docker build output:

services:
  dotnetcoremvctest:
    build:
      context: C:\Users\chris.claxton\source\repos\DotNetCoreMvcTest
      dockerfile: DotNetCoreMvcTest/Dockerfile
      target: base
    entrypoint: tail -f /dev/null
    environment:
      ASPNETCORE_ENVIRONMENT: Development
      ASPNETCORE_HTTPS_PORT: '44348'
      ASPNETCORE_URLS: https://+:443;http://+:80
      DOTNET_USE_POLLING_FILE_WATCHER: '1'
      NUGET_FALLBACK_PACKAGES: /root/.nuget/fallbackpackages
    image: dotnetcoremvctest:dev
    labels:
      com.microsoft.visualstudio.debuggee.arguments: ' --additionalProbingPath /root/.nuget/packages
        --additionalProbingPath /root/.nuget/fallbackpackages  bin/Debug/netcoreapp2.2/DotNetCoreMvcTest.dll'
      com.microsoft.visualstudio.debuggee.killprogram: /bin/bash -c "if PID=$$(pidof
        -x dotnet); then kill $$PID; fi"
      com.microsoft.visualstudio.debuggee.program: dotnet
      com.microsoft.visualstudio.debuggee.workingdirectory: /app
    ports:
    - published: 50286
      target: 80
    - published: 44348
      target: 443
    volumes:
    - C:\Users\username\source\repos\DotNetCoreMvcTest\DotNetCoreMvcTest:/app:rw
    - C:\Users\username\vsdbg\vs2017u5:/remote_debugger:ro
    - C:\Users\username\AppData\Roaming\ASP.NET\Https:/root/.aspnet/https:ro
    - C:\Users\username\AppData\Roaming\Microsoft\UserSecrets:/root/.microsoft/usersecrets:ro
    - C:\Program Files\dotnet\sdk\NuGetFallbackFolder:/root/.nuget/fallbackpackages:ro
    - C:\Users\username\.nuget\packages:/root/.nuget/packages:ro

The contents of the secret file are the above minus what is already in the docker-compose file:

version: '3.4'

services:
  dotnetcoreapitest:
    image: ${DOCKER_REGISTRY-}dotnetcoreapitest
    build:
      context: .
      dockerfile: DotNetCoreApiTest/Dockerfile
kdcllc commented 5 years ago

@chrisckc and @shanselman,

Unfortunately, I am running into situations where existing projects when reopen in Visual Studio.NET don't re-create secret.json file and also don't generate the Dev SSL Certificate. I made a Powershell script to fix that issue can be found here

This script also enables SSL certificates for VSCode on Windows dev boxes.

garfbradaz commented 5 years ago

This issue needs to be a official MS Document and a standardised approach for Windows and Linux Containers, using a variety of methods to orchestrate your containers (Like docker-compose).

inpicksys commented 5 years ago

So, dot it from scratch, DockerDotNET.sample, project created, compiled, cert created. Great. Run visual studio docker-compose, Wau -its worked via https: even browsed. Next time its broken, you can add it to trash. image this is after generating user secrets with correct password which was added to appettings.Development.json also added to docker-compose.override.yml image

And its sometimes accept password, next time already not. And even everything this Kestrel passes -> its not browsable. Its not working on local machine. I'm already not talking about deployment on remote machine. Its absolutely headache. Anybody have some stable solution that with corresponding steps, code samples, advice? This config hell. I can't load my UI, to just develop application, add features, services. I'm just trying some configs, waiting minutes & hours to get t load to see that it not worked.

So. I have 4 web app that should work under https on different port under linux docker containers, main tool is visual studio, latest version, netcore latest version. Will be appreciated for help. I'm in despair...

kdcllc commented 5 years ago

@inpicksys, I ran into this situation all the time try using my script to reset password and cert, https://github.com/kdcllc/win10andwsldev/blob/master/vsnet-docker-ssl-issue.md

garfbradaz commented 5 years ago

@kdcllc : Your script has saved me a shed load of time, good work there mate.

inpicksys commented 5 years ago

@kdcllc Thanks. I'm used your scripts. But docker containers are not started under linux. Kestrel not started. Googling doesn't help. So, as this is only for dev purposes, only way is just refuse Kestrel with https, as its take too much time to handle this. Will dig in nginx/apache proxy with https as its for production. Production & dev will match. Hope to get it worked with nginx. Thanks.

chrisckc commented 5 years ago

@inpicksys I notice you have "KestrelCertificatesDevelopment__Path specified which is the same as setting "Kestrel:Certificates:Development:Path" in my example.

This breaks the config due to a bug as i have described here: https://github.com/aspnet/Docs/issues/6199#issuecomment-418194220

So your issue may likely to be a result of the certificate not being found, the error is probably misleading as usual.

The only way to get it work work using the development config is to have the cert in the default path with the default name (same as the project name). If you look at the source code you will see why its broken, i described the problem here: https://github.com/aspnet/Docs/issues/6199#issuecomment-427834083

You can use "Kestrel:Certificates:Default:Path" just fine, but you have to name your "ASPNETCORE_ENVIRONMENT" env var something other than "Development"

bilalmalik777 commented 5 years ago

@bilalMlaik this documentatiom might be helpful Hosting ASP.NET Core Images with Docker over HTTPS.

I ran into situation where if I am using dotnet dev cert. I have to add manually the following to the apps secret:

  "Kestrel:Certificates:Development:Password": "43f5d164-3813-4dba-8e34-6337096f2df0"

The id must match the project secret entry:

    <UserSecretsId>8c5d84b3-27c0-4964-a32b-91dd9b1c01d3</UserSecretsId>

@kdcllc , I did all these things but nothing achieved. I am still facing same error in case of HTTPS.