"get_certificate tailscale" forces "on_demand_tls" global option

[TurboSlayer]

Backstory: I have a docker/caddy setup where I use a docker volume to expose the tailscaled socket to Caddy. It randomly stopped working today, so I tried to fix it. It looked like there were recent changes with socket paths so I tried different combinations with /tmp and /run but nothing worked. (EDIT: the error for this is caddy_vaultwarden | {"level":"debug","ts":1706154566.203921,"logger":"http.stdlib","msg":"http: TLS handshake error from 100.107.69.109:41312: no certificate available for 'redacted.x.ts.net'"})

The problem: In order to debug this I tried enabling get_certificate tailscale under the tls directive, but then it starts enforcing the "ask" endpoint which doesn't make sense for tailscale users. An example of the error is here.

I believe the "ask" enforcing is a fairly recent change, and so this may have been overlooked when the change was implemented. I know the docker volume approach isn't officially supported but it'd be great if this issue could be resolved so that I can try and debug it myself. Thanks!

[francislavoie]

What version are you using? What's the error message you see, exactly? Show your logs & config.

[TurboSlayer]

Yep, my bad, I was in a rush yesterday. Here's the info: Version: caddy:latest on docker (v2.7.6)

Caddyfile:

{
    debug
}

:443 {
  log {
    level INFO
    output file {$LOG_FILE} {
      roll_size 10MB
      roll_keep 10
    }
  }
  tls {
    get_certificate tailscale # Didn't need this before (ever since the feature was released)
  }

   encode gzip

   reverse_proxy vaultwarden:80
   reverse_proxy /notifications/hub vaultwarden:3012
}

Error:

caddy_vaultwarden | {"level":"info","ts":1706153302.1186092,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
caddy_vaultwarden | {"level":"warn","ts":1706153302.1194217,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
caddy_vaultwarden | {"level":"info","ts":1706153302.1199346,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//[::1]:2019","//127.0.0.1:2019","//localhost:2019"]}
caddy_vaultwarden | {"level":"info","ts":1706153302.1200917,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0006f5880"}
caddy_vaultwarden | {"level":"info","ts":1706153302.1201015,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc0006f5880"}
caddy_vaultwarden | Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 0: on-demand TLS cannot be enabled without an 'ask' endpoint to prevent abuse; please refer to documentation for details
caddy_vaultwarden | {"level":"info","ts":1706153303.7275922,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
caddy_vaultwarden | {"level":"warn","ts":1706153303.7283955,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
caddy_vaultwarden | {"level":"info","ts":1706153303.7289453,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy_vaultwarden | {"level":"info","ts":1706153303.729059,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000789300"}
caddy_vaultwarden | {"level":"info","ts":1706153303.729068,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000789300"}
caddy_vaultwarden | Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 0: on-demand TLS cannot be enabled without an 'ask' endpoint to prevent abuse; please refer to documentation for details
caddy_vaultwarden exited with code 1

EDIT: I have managed to work around this using the solution given in #5986.

[filiabel]

Hi!

Just want to report the same issue as it is still open, but using get_certificate http and not tailscale.

Previously I used tls cert key with a separate cron job fetching fresh wildcard cert and reloading Caddy at a given interval before trying out a new config using get_certificate. After config rewrite and trying to reload it fails with the same error message as above.

It works with the work-around as mentioned in #5986.

Providers cert endpoint returns 200 OK as expected.

Version v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A= Running as service on Ubuntu 22.04 LTS.

Config

{
        debug
}

*.mydomain.com {
        tls {
                # Extra query params as specified by provider
                get_certificate http https://cert.myprovider.com?domain=*.mydomain.com&secret=SECRET
        }

        log {
                format json
                output file /var/log/caddy/wildcard.log
        }

        @app1 host app1.mydomain.com
        handle @app1 {
                reverse_proxy localhost:3000
        }

        respond "No service running on {host}, yet...."
}

Log on caddy reload

{
  "level": "error",
  "ts": 1712191729.55823,
  "logger": "admin.api",
  "msg": "request error",
  "error": "loading config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: on-demand TLS cannot be enabled without an 'ask' endpoint to prevent abuse; please refer to documentation for details",
  "status_code": 400
}

Thanks for the great work you are doing with Caddy 👏

[mholt]

@filiabel I believe I have just fixed this in #6229