Closed xmonader closed 5 years ago
This behavior also appears with a Traefik as backend instead of caddy:
version: '3'
services:
edge:
image: traefik:v2.0.0-alpha4
command:
- "--configfile=/traefik.toml"
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- "./traefik.toml:/traefik.toml"
backend1:
image: traefik:v1.7.11
command:
- "--entryPoints=Name:https Address::443 TLS"
- "--defaultEntryPoints=https"
- "--acme.entryPoint=https"
- "--acme.email=damien@containo.us"
- "--acme.storage=acme.json"
- "--acme.tlsChallenge=true"
- "--acme.onHostRule=true"
- "--acme.acmeLogging=true"
- "--acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--docker.exposedByDefault=false"
- "--api"
labels:
- "traefik.enable=true"
- "traefik.frontend.rule=Host:YYYYYYYY"
- "traefik.port=8080"
ports:
- "8443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
backend2:
image: traefik:v1.7.11
command:
- "--entryPoints=Name:https Address::443 TLS"
- "--defaultEntryPoints=https"
- "--acme.entryPoint=https"
- "--acme.email=damien@containo.us"
- "--acme.storage=acme.json"
- "--acme.tlsChallenge=true"
- "--acme.onHostRule=true"
- "--acme.acmeLogging=true"
- "--acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--docker.exposedByDefault=false"
- "--api"
labels:
- "traefik.enable=true"
- "traefik.frontend.rule=Host:XXXXXXXXX"
- "traefik.port=8080"
ports:
- "8444:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
and traefik.toml:
[global]
debug = true
logLevel = "DEBUG"
[entryPoints]
[entryPoints.tcp]
address = "0.0.0.0:443"
[api]
[providers.file]
[tcp]
[tcp.services]
[tcp.services.caddy1.LoadBalancer]
[[tcp.services.caddy1.LoadBalancer.servers]]
address = "172.17.0.1:8443"
[tcp.services.caddy2.LoadBalancer]
[[tcp.services.caddy2.LoadBalancer.servers]]
address = "172.17.0.1:8444"
[tcp.routers]
[tcp.routers.caddy1]
entrypoints = ["tcp"]
rule = "HostSNI(`YYYYYY`)"
service = "caddy1"
[tcp.routers.caddy1.tls]
passthrough=true
[tcp.routers.caddy2]
entrypoints = ["tcp"]
rule = "HostSNI(`XXXXXXXX`)"
service = "caddy2"
[tcp.routers.caddy2.tls]
passthrough=true
However, after a while, the LE certificates are generated, even if we have the same error.
We are trying to dig deeper.
Hi @xmonader ,
I was not able to reproduce the behavior at all.
docker-compose.yml:
version: '3'
services:
edge:
image: traefik:v2.0.0-alpha4
command:
- "--configfile=/traefik.toml"
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- "./traefik.toml:/traefik.toml"
backend1:
image: abiosoft/caddy
ports:
- "8443:443"
environment:
- "ACME_AGREE=true"
volumes:
- ./Caddyfile-1:/etc/Caddyfile
backend2:
image: abiosoft/caddy
ports:
- "8444:443"
environment:
- "ACME_AGREE=true"
volumes:
- ./Caddyfile-2:/etc/Caddyfile
Caddyfile-1
https://XXXX {
bind 0.0.0.0
tls damien@noip.org {
ca https://acme-staging-v02.api.letsencrypt.org/directory
}
}
Caddyfile-2
https://YYYY {
bind 0.0.0.0
tls damien@noip.org {
ca https://acme-staging-v02.api.letsencrypt.org/directory
}
}
traefik.toml
[global]
debug = true
logLevel = "DEBUG"
[entryPoints]
[entryPoints.tcp]
address = "0.0.0.0:443"
[api]
[providers.file]
[tcp]
[tcp.services]
[tcp.services.caddy1.LoadBalancer]
[[tcp.services.caddy1.LoadBalancer.servers]]
address = "172.17.0.1:8443"
[tcp.services.caddy2.LoadBalancer]
[[tcp.services.caddy2.LoadBalancer.servers]]
address = "172.17.0.1:8444"
[tcp.routers]
[tcp.routers.caddy1]
entrypoints = ["tcp"]
rule = "HostSNI(`XXXX`)"
service = "caddy1"
[tcp.routers.caddy1.tls]
passthrough=true
[tcp.routers.caddy2]
entrypoints = ["tcp"]
rule = "HostSNI(`YYYY`)"
service = "caddy2"
[tcp.routers.caddy2.tls]
passthrough=true
We would need more informations from your side to help us reproducing and understand better what is happening: topology of your infrastructure, logs from Caddy, full reproduction case we could run on our own etc...
Aside, is there a particular reason to no use Traefik's Let's Encrypt capability, so it would simplify your setup (and use Caddy only a webserver)?
I'll try to reproduce and get more logs tomorrow,
for my use case I need to use traefik on a public IP as TCP proxy and forward the TLS traffic to some secure applications based on the SNI and they do the certificate generation, TLS termination not traefik.
I'm using caddy as an example of a secure application to simplify the setup and check if it works with traefik, because i already tested the flow against HAProxy and it worked.
for the infrasturture :
testbots.grid.tf
to traefik public IPtcp proxy
with public IPfirst.mybot.testbots.grid.tf
second.mybot.testbots.grid.tf
I tried multiple times, but can't reproduce it, but it's worrying for me to use it with such a random error.
Hi @xmonader , thanks for your feedbacks. We retried ~50 times each setup (with caddy and with Traefik), without any problem: certificates where always generated. Sometime the "403" message from LE appeared with no consequence on the certificate generation.
We were able to have issue for certificate generation when only the backend where started: it means your issue appeared during a time window where Caddy contacted Let's Encrypt while Traefik v2 did not had its configuration fully loaded, hence the default certificate.
If you ensure that Traefik v2 is started AND configured before any backend start, then you will face no error.
As you were not able to reproduce the issue, we're gonna close this issue assuming the technical issue is no more. If you happen to reproduce and can provide us how to reproduce, feel free to re-open it.
Thanks for your interest!
I've a machine with 1 public IP (traefik tcp proxy) that will serve multiple caddy applications over TLS
Do you want to request a feature or report a bug?
Bug
Did you try using a 1.7.x configuration for the version 2.0?
What did you do?
Setup caddy servers
172.18.0.35:443
172.18.0.44:443
e.g configuration
Setup Traefik
What did you expect to see?
In caddy side it should get the certificate without any problems
What did you see instead?
Got this very weird output
Output of
traefik version
: (What version of Traefik are you using?)What is your environment & configuration (arguments, toml, provider, platform, ...)?
Ubuntu 18.04 containers and here's the traefik toml config file
If applicable, please paste the log output in DEBUG level (
--log.level=DEBUG
switch)Traefik logs
Caddy let's encrypt suspicious output
Randomly running it worked somehow