LXD proxy concept

[fred-gb]

Hello, Very sorry to post here, I sign up to community but never receive mail. In spam, nothing.

I work on my little concept and I really want to use caddy, It's inspire me.

LXD Server container01 with proxy device on host to redirect 80 and 443 requests to container01 container01, I want to install caddy as reverse proxy. container02: wordpress, with caddy container03: rocketchat, with caddy etc.

How to plan this?

I tried on container01 to install caddy2 and create simple caddyfile:

xxxxxxxx.com
reverse_proxy wordpress01:80

I have this in systemd log:

Mar 14 07:52:44 container01 caddy[848]: 2020/03/14 07:52:44 [INFO][xxxxxxx.com] Obtain: Lock acquired; proceeding...
Mar 14 07:52:45 container01 caddy[848]: 2020/03/14 07:52:45 [INFO][xxxxxxx.com] Waiting on rate limiter...
Mar 14 07:52:45 container01 caddy[848]: 2020/03/14 07:52:45 [INFO][xxxxxxx.com] Done waiting
Mar 14 07:52:45 container01 caddy[848]: 2020/03/14 07:52:45 [INFO] [xxxxxxx.com] acme: Obtaining bundled SAN certificate given a CSR
Mar 14 07:52:46 container01 caddy[848]: 2020/03/14 07:52:46 [INFO] [xxxxxxx.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/3350457371
Mar 14 07:52:46 container01 caddy[848]: 2020/03/14 07:52:46 [INFO] [xxxxxxx.com] acme: Could not find solver for: tls-alpn-01
Mar 14 07:52:46 container01 caddy[848]: 2020/03/14 07:52:46 [INFO] [xxxxxxx.com] acme: use http-01 solver
Mar 14 07:52:46 container01 caddy[848]: 2020/03/14 07:52:46 [INFO] [xxxxxxx.com] acme: Trying to solve HTTP-01
Mar 14 07:56:47 container01 caddy[848]: 2020/03/14 07:56:47.334        INFO        admin.api        received request        {"method": "GET", "uri": "/", "remote_addr": "127.0.0.1:507
Mar 14 07:57:18 container01 caddy[848]: 2020/03/14 07:57:18 http: TLS handshake error from 127.0.0.1:44558: no certificate available for 'localhost'

Where is my error? And after, on container02 where wordpress is, Will I install Caddy with which type? With TLS question? Who I start the first? Caddy reverse proxy or Caddy wordpress container?

Thanks to show me the way. I'm little bit lost... Sorry...

I use Caddy 2.0-beta17.

[mholt]

Thanks for opening an issue! We'll look into this.

It's not immediately clear to me what is going on, so I'll need your help to understand it better.

Ideally, we need to be able to reproduce the bug in the most minimal way possible. This allows us to write regression tests to verify the fix is working. If we can't reproduce it, then you'll have to test our changes for us until it's fixed -- and then we can't add test cases, either.

I've attached a template below that will help make this easier and faster! It will ask for some information you've already provided; that's OK, just fill it out the best you can. :+1:

I've also included some helpful tips below the template. Feel free to let me know if you have any questions!

Thank you again for your report, we look forward to resolving it!

Template

## 1. Environment

### 1a. Operating system and version

```
paste here
```

### 1b. Caddy version (run `caddy version` or paste commit SHA)

```
paste here
```

### 1c. Go version (if building Caddy from source; run `go version`)

```
paste here
```

## 2. Description

### 2a. What happens (briefly explain what is wrong)

### 2b. Why it's a bug (if it's not obvious)

### 2c. Log output

```
paste terminal output or logs here
```

### 2d. Workaround(s)

### 2e. Relevant links

## 3. Tutorial (minimal steps to reproduce the bug)

Helpful tips

1. Environment: Please fill out your OS and Caddy versions, even if you don't think they are relevant. (They are always relevant.) If you built Caddy from source, provide the commit SHA and specify your exact Go version.

2. Description: Describe at a high level what the bug is. What happens? Why is it a bug? Not all bugs are obvious, so convince readers that it's actually a bug.

   • 2c) Log output: Paste terminal output and/or complete logs in a code block. DO NOT REDACT INFORMATION except for credentials.
   • 2d) Workaround: What are you doing to work around the problem in the meantime? This can help others who encounter the same problem, until we implement a fix.
   • 2e) Relevant links: Please link to any related issues, pull requests, docs, and/or discussion. This can add crucial context to your report.

3. Tutorial: What are the minimum required specific steps someone needs to take in order to experience the same bug? Your goal here is to make sure that anyone else can have the same experience with the bug as you do. You are writing a tutorial, so make sure to carry it out yourself before posting it. Please:

   • Start with an empty config. Add only the lines/parameters that are absolutely required to reproduce the bug.
   • Do not run Caddy inside containers.
   • Run Caddy manually in your terminal; do not use systemd or other init systems.
   • If making HTTP requests, avoid web browsers. Use a simpler HTTP client instead, like curl.
   • Do not redact any information from your config (except credentials). Domain names are public knowledge and often necessary for quick resolution of an issue!
   • Note that ignoring this advice may result in delays, or even in your issue being closed. 😞 Only actionable issues are kept open, and if there is not enough information or clarity to reproduce the bug, then the report is not actionable.

Example of a tutorial:

Create a config file: ``` { ... } ``` Open terminal and run Caddy: ``` $ caddy ... ``` Make an HTTP request: ``` $ curl ... ``` Notice that the result is ___ but it should be ___.

[fred-gb]

1. Environment

1a. Operating system and version

Ubuntu 18.04 for host LXD
Ubuntu 18.04 minimal for LXC container

1b. Caddy version (run caddy version or paste commit SHA)

v2.0.0-beta.17 h1:x+Ur3uX83j+STerOWsrLDlknXe7z71VnO5xD+H2OwAw=

2. Description

2a. What happens (briefly explain what is wrong)

I cannot obtain TLS .

The final goal is: A main Caddy reverse-proxy who redirect to other caddy https server.

First step on main caddy. Before turn to reverse-proxy, try to make a simple HTTPS server.

2b. Why it's a bug (if it's not obvious)

2c. Log output

When I started by systemctl start/restart caddy

Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11.144        INFO        admin        admin endpoint started        {"address": "localhost:2019", "enforce_origin": false, "
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11.144        INFO        http        server is listening only on the HTTPS port but has no TLS connection policies; adding o
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11 [INFO][cache:0xc0005da690] Started certificate maintenance routine
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11.144        INFO        http        enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11.145        INFO        tls        cleaned up storage units
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11.145        INFO        http        enabling automatic TLS certificate management        {"domains": ["domain.com"]}
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11.145        INFO        autosaved config        {"file": "/var/lib/caddy/.config/caddy/autosave.json"}
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11.145        INFO        serving initial configuration
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11 [INFO][domain.com] Obtain certificate; acquiring lock...
Mar 14 21:43:11 container01 caddy[1662]: 2020/03/14 21:43:11 [INFO][domain.com] Obtain: Lock acquired; proceeding...
Mar 14 21:43:31 container01 caddy[1662]: 2020/03/14 21:43:31 [ERROR] Making new ACME client: get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get "https://acme-v02.a
Mar 14 21:43:52 container01 caddy[1662]: 2020/03/14 21:43:52 [ERROR] Making new ACME client: get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get "https://acme-v02.a
Mar 14 21:43:53 container01 caddy[1662]: 2020/03/14 21:43:53 [ERROR] attempt 1: [domain.com] Obtain: get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get "https:
Mar 14 21:45:13 container01 caddy[1662]: 2020/03/14 21:45:13 [ERROR] Making new ACME client: get directory at 'https://acme-staging-v02.api.letsencrypt.org/directory': Get "https://ac
Mar 14 21:45:34 container01 caddy[1662]: 2020/03/14 21:45:34 [ERROR] Making new ACME client: get directory at 'https://acme-staging-v02.api.letsencrypt.org/directory': Get "https://ac
Mar 14 21:45:35 container01 caddy[1662]: 2020/03/14 21:45:35 [ERROR] attempt 2: [domain.com] Obtain: get directory at 'https://acme-staging-v02.api.letsencrypt.org/directory': Get

When I started by root@container01:/var/lib/caddy# caddy run --config /etc/caddy/Caddyfile

root@container01:/var/lib/caddy# caddy run --config /etc/caddy/Caddyfile
2020/03/14 21:50:13.868 INFO    using provided configuration    {"config_file": "/etc/caddy/Caddyfile", "config_adapter": ""}
2020/03/14 21:50:13.870 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["localhost:2019"]}
2020/03/14 21:50:13.871 INFO    http    server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
2020/03/14 21:50:13.871 INFO    http    enabling automatic HTTP->HTTPS redirects    {"server_name": "srv0"}
2020/03/14 21:50:13.872 INFO    tls cleaned up storage units
2020/03/14 21:50:13.872 INFO    http    enabling automatic TLS certificate management   {"domains": ["domain.com"]}
2020/03/14 21:50:13.872 INFO    autosaved config    {"file": "/root/.config/caddy/autosave.json"}
2020/03/14 21:50:13.872 INFO    serving initial configuration
2020/03/14 21:50:13 [INFO][cache:0xc000616050] Started certificate maintenance routine
2020/03/14 21:50:13 [INFO][domain.com] Obtain certificate; acquiring lock...
2020/03/14 21:50:13 [INFO][domain.com] Obtain: Lock acquired; proceeding..
2020/03/14 21:50:33 [ERROR] Making new ACME client: get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get "https://acme-v02.api.letsencrypt.org/directory": dial tcp: lookup acme-v02.api.letsencrypt.org on 127.0.0.53:53: read udp 127.0.0.1:34585->127.0.0.53:53: i/o timeout (attempt 1/2)
2020/03/14 21:50:54 [ERROR] Making new ACME client: get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get "https://acme-v02.api.letsencrypt.org/directory": dial tcp: lookup acme-v02.api.letsencrypt.org on 127.0.0.53:53: read udp 127.0.0.1:51636->127.0.0.53:53: i/o timeout (attempt 2/2)
2020/03/14 21:50:55 [ERROR] attempt 1: [domain.com] Obtain: get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get "https://acme-v02.api.letsencrypt.org/directory": dial tcp: lookup acme-v02.api.letsencrypt.org on 127.0.0.53:53: read udp 127.0.0.1:51636->127.0.0.53:53: i/o timeout - retrying in 1m0s (42.037291427s/720h0m0s elapsed)...
2020/03/14 21:52:15 [ERROR] Making new ACME client: get directory at 'https://acme-staging-v02.api.letsencrypt.org/directory': Get "https://acme-staging-v02.api.letsencrypt.org/directory": dial tcp: lookup acme-staging-v02.api.letsencrypt.org on 127.0.0.53:53: read udp 127.0.0.1:58336->127.0.0.53:53: i/o timeout (attempt 1/2)

curl -i https://localhost

curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

2d. Workaround(s)

No workaround :'(

3. Tutorial (minimal steps to reproduce the bug)

My ansible playbook to install caddy:

- hosts: container01
  become: yes
  vars:
    caddy_git_url: "https://github.com/caddyserver/caddy/releases/download/v2.0.0-beta.17/caddy2_beta17_linux_amd64"
  tasks:
    - name: Caddy | Les paquets
      include_tasks: ../../ansible-tasks/linux/system/apt_install.yml
      vars:
        packages: libnss3-tools

    - name: creation du group et user caddy
      include_tasks: ../../ansible-tasks/linux/system/useradd.yml
      vars:
        username: caddy
        user_groups: caddy
        user_homedir: "/var/lib/caddy"
        system: yes

    - name: Creation repertoire de config
      file:
        path: /etc/caddy/
        state: directory
        recurse: yes
        owner: caddy
        group: caddy

    - name: Creation repertoire de /var/www
      file:
        path: /var/www/
        state: directory
        recurse: yes
        owner: caddy
        group: caddy  

    - name: Verif si la config est deja presente
      stat:
        path: /etc/caddy/Caddyfile
      register: caddy_config    

    - name: Creation du fichier de conf vierge de caddy avant de se faire remplir
      template:
        src: "caddy/Caddyfile.j2"
        dest: "/etc/caddy/Caddyfile"
        owner: caddy
        group: caddy   
      when: caddy_config.stat.exists == false     

    - name: Recuperation du binary caddy
      get_url:
        url: "{{ caddy_git_url }}"
        dest: "/usr/bin/caddy"
        owner: caddy
        group: caddy  
        mode: '0755'

    - name: Recuperation du systemd.service
      template:
        src: "caddy/caddy.service.j2"
        dest: "/etc/systemd/system/caddy.service"
      notify: start caddy

  handlers:
     - name: start caddy
       systemd:
         state: started
         name: caddy
         enabled: yes 

My Caddyfile:

domain.com
root * /var/www/

On host:

lxc config device show haproxy01
http_port:
  connect: tcp:127.0.0.1:80
  listen: tcp:0.0.0.0:80
  type: proxy
https_port:
  connect: tcp:127.0.0.1:443
  listen: tcp:0.0.0.0:443
  type: proxy

In Firefox: PR_END_OF_FILE_ERROR

Do you have enough infos? Thanks a lot to support. Hope to reach my goal with caddy!

[fred-gb]

Evolution. My firts error. Shame on me. Problem with LXD Bridge, no network reachable. After reboot snap.lxd.daemon. Network works.

Now, the log, with systemctl start caddy

Mar 14 22:27:42 container01 caddy[215]: 2020/03/14 22:27:42 [INFO] [domain.com] acme: Could not find solver for: tls-alpn-01
Mar 14 22:27:42 container01 caddy[215]: 2020/03/14 22:27:42 [INFO] [domain.com] acme: use http-01 solver
Mar 14 22:27:42 container01 caddy[215]: 2020/03/14 22:27:42 [INFO] [domain.com] acme: Trying to solve HTTP-01
Mar 14 22:27:42 container01 caddy[215]: 2020/03/14 22:27:42 [INFO][domain.com] Served key authentication (HTTP challenge)
Mar 14 22:27:42 container01 caddy[215]: 2020/03/14 22:27:42 [INFO][domain.com] Served key authentication (HTTP challenge)
Mar 14 22:27:42 container01 caddy[215]: 2020/03/14 22:27:42 [INFO][domain.com] Served key authentication (HTTP challenge)
Mar 14 22:27:42 container01 caddy[215]: 2020/03/14 22:27:42 [INFO][domain.com] Served key authentication (HTTP challenge)
Mar 14 22:27:50 container01 caddy[215]: 2020/03/14 22:27:50 http: TLS handshake error from 127.0.0.1:44774: no certificate available for 'domain.com'
Mar 14 22:27:51 container01 caddy[215]: 2020/03/14 22:27:51 http: TLS handshake error from 127.0.0.1:44778: no certificate available for 'domain.com'
Mar 14 22:28:24 container01 caddy[215]: 2020/03/14 22:28:24 [INFO] [domain.com] The server validated our request
Mar 14 22:29:11 container01 caddy[215]: 2020/03/14 22:29:11 http: TLS handshake error from 127.0.0.1:44782: no certificate available for 'domain.com'
Mar 14 22:29:11 container01 caddy[215]: 2020/03/14 22:29:11 http: TLS handshake error from 127.0.0.1:44784: no certificate available for 'domain.com'
Mar 14 22:29:14 container01 caddy[215]: 2020/03/14 22:29:14 http: TLS handshake error from 127.0.0.1:44786: no certificate available for 'domain.com'
Mar 14 22:29:15 container01 caddy[215]: 2020/03/14 22:29:15 http: TLS handshake error from 127.0.0.1:44788: no certificate available for 'domain.com'

With root user, caddy run --config /etc/caddy/Caddyfile

root@container01:~# caddy run --config /etc/caddy/Caddyfile
2020/03/14 22:32:03.436 INFO    using provided configuration    {"config_file": "/etc/caddy/Caddyfile", "config_adapter": ""}
2020/03/14 22:32:03.440 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["localhost:2019"]}
2020/03/14 22:32:03.440 INFO    http    server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
2020/03/14 22:32:03.440 INFO    http    enabling automatic HTTP->HTTPS redirects    {"server_name": "srv0"}
2020/03/14 22:32:03 [INFO][cache:0xc0004b1680] Started certificate maintenance routine
2020/03/14 22:32:03.440 INFO    tls cleaned up storage units
2020/03/14 22:32:03.441 INFO    http    enabling automatic TLS certificate management   {"domains": ["domain.com"]}
2020/03/14 22:32:03.441 INFO    autosaved config    {"file": "/root/.config/caddy/autosave.json"}
2020/03/14 22:32:03.441 INFO    serving initial configuration
2020/03/14 22:32:03 [INFO][domain.com] Obtain certificate; acquiring lock...
2020/03/14 22:32:03 [INFO][domain.com] Obtain: Lock acquired; proceeding...
2020/03/14 22:32:04 [INFO][domain.com] Waiting on rate limiter...
2020/03/14 22:32:04 [INFO][domain.com] Done waiting
2020/03/14 22:32:04 [INFO] [domain.com] acme: Obtaining bundled SAN certificate given a CSR
2020/03/14 22:32:05 [INFO] [domain.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/3361542453
2020/03/14 22:32:05 [INFO] [domain.com] acme: use tls-alpn-01 solver
2020/03/14 22:32:05 [INFO] [domain.com] acme: Trying to solve TLS-ALPN-01
2020/03/14 22:32:05 http: TLS handshake error from 127.0.0.1:44796: EOF
2020/03/14 22:32:06 [INFO][domain.com] Served key authentication certificate (TLS-ALPN challenge)
2020/03/14 22:32:06 [INFO][domain.com] Served key authentication certificate (TLS-ALPN challenge)
2020/03/14 22:32:06 [INFO][domain.com] Served key authentication certificate (TLS-ALPN challenge)
2020/03/14 22:32:25 [INFO][domain.com] Served key authentication certificate (TLS-ALPN challenge)
2020/03/14 22:32:48 [INFO] [domain.com] The server validated our request
2020/03/14 22:32:48 [INFO] [domain.com] acme: Validations succeeded; requesting certificates
2020/03/14 22:32:49 [INFO] [domain.com] Server responded with a certificate.
2020/03/14 22:32:49 [INFO][domain.com] Certificate obtained successfully
2020/03/14 22:32:49 [INFO][domain.com] Obtain: Releasing lock

It's work! (But I have a white page where I have a small html page)

So where is the read/write access to set? I read in doc that ask for password. For what et where? I need to know to integrate to ansible and simply works with systemd and caddy user.

Thanks

[mholt]

Thanks for looking into it more!

A blank white page usually just means that none of the routes wrote a response, or the response was empty. At this point I would look into your backend config and make sure it's all correct!

[fred-gb]

Hello, Thanks but need more infos too please.

About why when I run in root it's work, but don't when I run with systemd and caddy user. Where I can set write permission? Thanks

[fred-gb]

Caddy reverse_proxy container:

http://domain.com {
reverse_proxy wordpress01:80
}

https://domain.com {
reverse_proxy wordpress01:443
}

And backend container Caddyfile

domain.com
root * /var/www/
file_server

exec ll /var/www/

total 3
drwxr-xr-x  2 caddy caddy   3 Mar 13 22:42 ./
drwxr-xr-x 14 root  root   16 Mar 13 22:09 ../
-rw-r--r--  1 caddy caddy 136 Mar 13 22:42 index.htm

And always I need to launch with caddy run, with systemd, it's not working. Thanks

[fred-gb]

When I check with my web browser, I have, on reverse proxy log:

2020/03/15 19:23:31.103 ERROR   http.log.error  remote error: tls: internal error   {"request": {"method": "GET", "uri": "/", "proto": "HTTP/2.0", "remote_addr": "127.0.0.1:46214", "host": "domain.com", "headers": {"User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:73.0) Gecko/20100101 Firefox/73.0"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"], "Accept-Language": ["fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3"], "Accept-Encoding": ["gzip, deflate, br"], "Dnt": ["1"], "Upgrade-Insecure-Requests": ["1"], "Te": ["trailers"]}, "tls": {"resumed": false, "version": 772, "ciphersuite": 4865, "proto": "h2", "proto_mutual": true, "server_name": "domain.com"}}, "status": 502, "err_id": "n68dpxm43", "err_trace": "reverseproxy.(*Handler).ServeHTTP (reverseproxy.go:363)"}

On backend container log:

2020/03/15 19:23:31 http: TLS handshake error from 10.244.32.196:34740: no certificate available for 'wordpress01'

[mohammed90]

To fix the error of no certificate available for 'wordpress01', change

http://domain.com {
reverse_proxy wordpress01:80
}

https://domain.com {
reverse_proxy wordpress01:443
}

To (change 443 to 80)

http://domain.com {
reverse_proxy wordpress01:80
}

https://domain.com {
reverse_proxy wordpress01:80
}

As for running it with user other than root, I'm not sure how LXD works, but the directive AmbientCapabilities=CAP_NET_BIND_SERVICE is needed in systemd service files. You might want to look at the systemd unit file in our dist repo.

[fred-gb]

Hello and thanks but... Not working for change 443 to 80, and now, I don't have log anymore, just in my browser: The page is not redirected correctly

And for systemd, it's seems to work. Thanks for that.

[mohammed90]

Ok. At this point, this is not a bug report. Can you make a post in the forum https://caddy.community . Include details of wordpress01 configuration (is it listening on :443, among other stuff).

[fred-gb]

I would like, but signup not working for me. I try with to register by mail and GitHub, each time, the mail verification never comes. I check many times spam, but nothing. If admin of community can check my subscription and validate it. Thanks

[mholt]

@quanticware What's your username / email address? I don't think I see you in any of the logs... I am looking in both the forum logs and the email sender's logs but I'll need to know the email address to try to learn more.

[fred-gb]

@mholt Send by mail. Thanks

[mholt]

@quanticware -- thanks; your mail server's spam settings are a bit aggressive:

The quanticware.com server blocked this message from being delivered. While we won't try to send this message again, we will attempt to send new messages to this address in the future.

You can view all addresses in your Blocks suppression group. Learn more.

Full response from the quanticware.com server:

550 5.2.0 Spam message rejected

[fred-gb]

ok, but it's my provider, because I disable spam filter in my account. And know I forget my password... Sorry... What is mail where you use to community, I will ask my provider. Thanks

[mholt]

The emails come from noreply@caddy.community

[fred-gb]

My provider needs the raw headers of mail. Thanks

[mholt]

@quanticware I've found your account on the forums and tried setting the trust level to 1 manually, see if that works. Your username is Fred.

[fred-gb]

Thanks, but I don't remember of password I set at sign up, so, same problem if I ask new password I'll don't receive mail. I sent you a mail because I tried with another mail and it does'nt work too. Very sorry...