search

docksal / service-vhost-proxy

Virtual host proxy service image for Docksal
http://docksal.io
MIT License
7 stars 14 forks source link

Support request - Default and custom certs for HTTPS #34

Closed frederickjh closed 5 years ago

frederickjh commented 6 years ago

Hi! The directions in the README.md state in the section Default and custom certs for HTTPS:

To use custom certs, mount a folder with certs to /etc/certs/custom

In which container do these need mounted?

I am guessing that the label io.docksal.cert-name=dev.example.com would go in docksal-local.yml for each project. Is that correct? Is this under, services, web?

Where should the environmental variables get set? Like DEFAULT_CERT and ACCESS_LOG.

Sorry I am just not putting all the pieces together from the README.md.

Thanks for your time and help! Frederick

lmakarov commented 6 years ago

The instructions in readme are written in an abstracted way. For most users, fin can handle the setup automatically. Until released, you would have to use the develop Docksal/fin version (DOCKSAL_VERSION=develop fin update), then:

Here's an example of my certs folder:

$ ls -la $HOME/.docksal/certs
total 16
drwxr-xr-x   7 leonid  staff   224 Jul 17 14:44 .
drwxr-xr-x  15 leonid  staff   480 Oct  2 09:34 ..
-rw-r--r--   1 leonid  staff  1671 Jul  2 10:47 example.com.crt
-rw-r--r--   1 leonid  staff  3264 Jul  2 10:47 example.com.key
drwxr-xr-x   7 leonid  staff   224 Jul 17 14:44 x.docksal.io
lrwxr-xr-x   1 leonid  staff    26 Jul 17 14:27 x.docksal.io.crt -> x.docksal.io/fullchain.pem
lrwxr-xr-x   1 leonid  staff    24 Jul 17 14:28 x.docksal.io.key -> x.docksal.io/privkey.pem

Here's the relevant code in fin, that starts the vhost-proxy: https://github.com/docksal/docksal/blob/bba17ce96264b698249d7c50f999121b2674851f/bin/fin#L3287-L3307

lmakarov commented 6 years ago

@frederickjh if you feel like instructions in the readme can be improved or expanded - please contribute with a PR :)

frederickjh commented 6 years ago

I would if I had any idea what I was actually suppose to do and where. I still do not so, I am afraid I cannot be of much help with the documentation until I understand what it all means.

lmakarov commented 6 years ago

@frederickjh I added the docs on variable mapping for configuring the vhost-proxy settings in ~/.docksal/docksal.env - https://github.com/docksal/service-vhost-proxy#variable-mapping-for-docksal

I think that was the missing piece. Let me know if that does or does not help it more clear.

frederickjh commented 6 years ago

@lmakarov that helps. I still do not know how to do this:

To use custom certs, mount a folder with certs to /etc/certs/custom.

Does the vhost-proxy service container have a .yml file some where to add this as a volume? Or is there another way to mount my custom certificates folder to /etc/certs/custom?

lmakarov commented 6 years ago

@frederickjh the certs go into $HOME/.docksal/certs on the host machine. Example in https://github.com/docksal/service-vhost-proxy/issues/34#issuecomment-426417396

frederickjh commented 6 years ago

@lmakarov I just tried this again and still cannot get it working. I put the certificates in the $HOME/.docksal/certs and named them after the domain. I ran fin system reset vhost-proxy I checked the log with docker logs docksal-vhost-proxy but did not see any mention of mounting the folder there. I was able to run bash in the docksal-vhost-proxy container and see my certificates in /etc/certs/custom.

I went to a running project and did fin restart I checked the log with fin logs web:

web_1      | [Sat Oct 20 20:29:49.228171 2018] [ssl:warn] [pid 1:tid 140438911216456] AH01906: web:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
web_1      | [Sat Oct 20 20:29:49.228209 2018] [ssl:warn] [pid 1:tid 140438911216456] AH01909: web:443:0 server certificate does NOT include an ID which matches the server name
web_1      | [Sat Oct 20 20:29:49.233103 2018] [ssl:warn] [pid 1:tid 140438911216456] AH01906: web:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
web_1      | [Sat Oct 20 20:29:49.233117 2018] [ssl:warn] [pid 1:tid 140438911216456] AH01909: web:443:0 server certificate does NOT include an ID which matches the server name
web_1      | [Sat Oct 20 20:29:49.251349 2018] [mpm_event:notice] [pid 1:tid 140438911216456] AH00489: Apache/2.4.27 (Unix) LibreSSL/2.5.4 configured -- resuming normal operations

Is the certificate in your certificate folder above a wildcard certificate (x.docksal.io.crt)? Is the x a wildcard? I tried mine with and without the x. in front of the domain name but it gives me the same messages as above each time. I am trying to install a wildcard certificate, which is what I would assume you are doing not making a new certificate for each developments site, but maybe I am wrong.

When I check the certificates I see:

X509v3 Subject Alternative Name: 
                DNS:*.dev.example.com, DNS:dev.example.com

So it looks like it should work as a wildcard certificate to me. I did do the DNS verification to get this certificate. I renamed instead of linked the privkey.pem and fullchain.pem from the Let's Encrypt files like you did. Thanks again for your time and help! Frederick

lmakarov commented 6 years ago

Your project's VIRTUAL_HOST setting should match the certificate you want to use.

E.g. for a project with VIRTUAL_HOST=example.com, vhost-proxy will try to load the following cert files (as seen inside the docksal-vhost-proxy container):

/etc/certs/custom/example.com.crt
/etc/certs/custom/example.com.key

You can check the resulting virtual host config using this command:

docker exec docksal-vhost-proxy cat /etc/nginx/conf.d/vhosts.conf

Example:

$ fin config get VIRTUAL_HOST
"test.x.docksal.io"

$ docker exec docksal-vhost-proxy cat /etc/nginx/conf.d/vhosts.conf
    # -------------------------------------------------- #
    # Reachable via "test_default" network
    upstream test-web {
        # test_web_1
        server 172.20.0.4:80;
    }
    ## HTTPS
    server {
        listen 80;
        listen 443 ssl http2;
        server_name test.x.docksal.io;
        server_name *.test.x.docksal.io;
        server_name test.x.docksal.io.*;
        ssl_certificate /etc/certs/custom/x.docksal.io.crt;
        ssl_certificate_key /etc/certs/custom/x.docksal.io.key;
        ssl_session_cache builtin:1000 shared:SSL:10m;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
        ssl_prefer_server_ciphers on;
        location / {
            proxy_pass http://test-web;
        }
    }
    # -------------------------------------------------- #

If there is no matching cert, then the default one will be used:

$ fin config get VIRTUAL_HOST
"test.docksal"

$ docker exec docksal-vhost-proxy cat /etc/nginx/conf.d/vhosts.conf
    # -------------------------------------------------- #
    # Reachable via "test_default" network
    upstream test-web {
        # test_web_1
        server 172.20.0.4:80;
    }
    ## HTTPS
    server {
        listen 80;
        listen 443 ssl http2;
        server_name test.docksal;
        server_name *.test.docksal;
        server_name test.docksal.*;
        ssl_certificate /etc/certs/server.crt;
        ssl_certificate_key /etc/certs/server.key;
        ssl_session_cache builtin:1000 shared:SSL:10m;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
        ssl_prefer_server_ciphers on;
        location / {
            proxy_pass http://test-web;
        }
    }
    # -------------------------------------------------- #35

In vhost-proxy v1.4 (upcoming release of Docksal) it is possible to override the default cert name DOCKSAL_VHOST_PROXY_DEFAULT_CERT global variable:

$ fin config set --global DOCKSAL_VHOST_PROXY_DEFAULT_CERT=example.com
...
$ fin system reset vhost-proxy
...

This overwrites the /etc/certs/server.crt and /etc/certs/server.key with the cert you configure. You will see the following in the vhost-proxy startup logs:

$ fin docker logs docksal-vhost-proxy
Using a custom default certificate: 'example.com'
...

Note: you will still see the following in vhost-proxy's nginx configuration:

$ docker exec docksal-vhost-proxy cat /etc/nginx/conf.d/vhosts.conf
...
        ssl_certificate /etc/certs/server.crt;
        ssl_certificate_key /etc/certs/server.key;
...
frederickjh commented 5 years ago

I just tested this as someone asked how to do it and it looks like after a reboot it is working so I am closing. Thanks to @lmakarov for your time and patience with me! Frederick

lmakarov commented 5 years ago

@frederickjh ok, cool. Do you feel like there are gaps in the docs that need to be addressed?

frederickjh commented 5 years ago

@lmakarov Sorry to take so long to get back to you on this. My Let's Encrypt wildcard certificates on my Docksal installation expired and I had to take another look at this. I have the following points that might be helpful to add to the documentation.

  1. Certificates in ~/.docksal/certs cannot be linked. Copy them in if they are generated elsewhere.
  2. If using Let's Encrypt generated certificates copy the following files to ones with your wildcard domain name. We will use dev.example.com as the domain name in this example.

Let's Encrypt File -> Docksal Certificate file

I also have one question in my $HOME/.docksal/docksal.env I have commented out the variable `CONFIG_CERTS=' I do not see it is the documentation, but I also do not remember where I got it from. Looks like it should be able to set the directory where the certificates will be located.

MatteoBorgognoni commented 5 years ago

I am still unable to make this work.

I have:

but the server is still loading the default certificate. If I ssh into the docksal-vhost-proxy container I can see that the custom folder hasn't been mounted

What am I missing?

docksal.yml `version: "2.1"

services: cli: environment:

docksal.env COMPOSE_PROJECT_NAME=xxx CLI_IMAGE='docksal/cli:2.4-php7.2 DOCKSAL_STACK=default DOCROOT=docroot MYSQL_DATABASE=xxx_db MYSQL_PASSWORD=xxx_pass MYSQL_USER=xxx_user XDEBUG_ENABLED=0 DOCKSAL_VHOST_PROXY_DEFAULT_CERT="xxx.docksal" VIRTUAL_HOST_CERT_NAME="xxx.docksal"

frederickjh commented 5 years ago

@MatteoBorgognoni Seeing that you have sanitized your setting in this issue it is hard to tell what exactly is wrong, however I would check that the certificate and key files are in the correct location and have the correct names. For your sanitized domain name the location and filenames would be:

/home/user/.docksal/certs/xxx.docksal.crt
/home/user/.docksal/certs/xxx.docksal.key

I am using a wildcard certificate so I placed the setting: DOCKSAL_VHOST_PROXY_DEFAULT_CERT= into /home/user/.docksal/docksal.env which is the global Docksal environment settings file.

I believe that VIRTUAL_HOST_CERT_NAME= should not go in that file but be used in a project's .docksal/docksal.env file to say do not use the default certificate.

The fact that it is falling back and using the default certificate means that something is not configured correctly.

MatteoBorgognoni commented 5 years ago

@frederickjh assuming my virtusl host server name is domain.docksal and I do not want to use wildcard certificates I have in my repo: .docksal/certs/domain.docksal.crt .docksal/certs/domain.docksal.key Then I have specified the variable VIRTUAL_HOST_CERT_NAME=domain.docksal

Seeing that that wasn't working no matter which command I would use to reset the vhost-proxy I also tried to add the DOCKSAL_VHOST_PROXY_DEFAULT_CERT variable without success.

ms-marcus commented 5 years ago

@MatteoBorgognoni Make sure the certs are in in ~/.docksal/certs and not in the repo, as the certs need to be attached to the vhost-proxy container and not the project containers.

After that, assuming VIRTUAL_HOST is set to myproject.docksal in .docksal/docksal.env within your repo root and your files are named ~/.docksal/certs/myproject.docksal.{crt,key} it should all work.

Make sure to run fin system reset vhost-proxy and then within your repo fin up to get everything back in line.

If it continues not to work, try posting the output of: docker exec docksal-vhost-proxy cat /etc/nginx/conf.d/vhosts.conf

Assuming you don't want to change the certs for any other project, there should be no need to set DOCKSAL_VHOST_PROXY_DEFAULT_CERT or VIRTUAL_HOST_CERT_NAME

frederickjh commented 5 years ago

The certificate nor the key files will never be in your web project repository, as Docksal does not look for them there. It only looks in the home folder of the user Docker run as. So if the user name is foo then it looks in /home/foo/.certs/ for the .crt and .key file that match the domain name, even if your are overriding the default certificate with VIRTUAL_HOST_CERT_NAME=domain.docksal in a Docksal project's .docksal/docksal.env file. In this case put the certificate and key files in:

/home/foo/.certs/domain.docksal.crt
/home/foo/.certs/domain.docksal.key

There is no need to set the default certificate if you are not using a wildcard certificate like I am.

After restarting the project you can check in in the container:

/etc/certs/server.crt
/etc/certs/server.key

to see if the key and certificate the you want to use have override the default ones.

frederickjh commented 5 years ago

You should also check the you are using the correct format for the files. If using LetsEncrypt to generate them, is my comments above about this.

acondura commented 3 years ago

I had problems with this too and after I tried all of the options in this thread, the one that finally solved it was the one explained by @lmakarov and given in the docs https://docs.docksal.io/stack/configuration-variables/#virtual-host-cert-name:

$HOME/.docksal/certs

that is: user's home (not project home) and then the '/.docksal/certs'. If your project root dir is in /home/[DOCKSAL_USER]/[PROJECT_NAME] then docksal will load the certs from:

I don't know why it became confusing but the docs say it quite clearly: $HOME/.docksal/certs. Perhaps a more explicit version would be [USER_HOME_DIR]/.docksal/certs. Just a thought. Anyway, I got it fixed with the above.