Acme parsing error

[petrleocompel]

Classification

• [x] Serious bug

Reproducibility

• [X] Always

Docker information

Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false

Server:
 Containers: 14
  Running: 14
  Paused: 0
  Stopped: 0
 Images: 26
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: /usr/libexec/docker/docker-init
 containerd version:
 runc version:
 init version:
 Security Options:
  seccomp
   Profile: builtin
  selinux
  cgroupns
 Kernel Version: 6.5.11-300.fc39.x86_64
 Operating System: Fedora CoreOS 39.20231119.3.0
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.6GiB
 Name: cac
 ID: 63493c27-f150-4b61-a3db-f6880880998b
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: petrleocompel
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: true

docker images mailserver2/mailserver --digests --filter "dangling=false"
REPOSITORY               TAG       DIGEST                                                                    IMAGE ID       CREATED        SIZE
mailserver2/mailserver   1.1.15    sha256:c85fc055d805333a18210fa6d8fc7227a2f0d3dff519b2cfa805bc0410b61c63   d7b64bc841d8   6 months ago   421MB

docker images traefik --digests --filter "dangling=false"
REPOSITORY   TAG       DIGEST                                                                    IMAGE ID       CREATED        SIZE
traefik      2.10      sha256:c5181ddf303f1ccfd4bd6d1d9c4867b0500efb6089a0f9ccb16612438f6e934f   64586c703ab1   5 weeks ago    153MB

Description

Wildcard letsencrypt certificate cannot throws error in parsing.

Steps to reproduce

1. Wildcard domain configuration with traefik:2.10

Expected results

Parsing correctly PEM

Actual results

Acme certificate is present in JSON but cannot be parsed..

Debugging information

[INFO] MariaDB/PostgreSQL hostname not found in /etc/hosts
[INFO] Container IP found, adding a new record in /etc/hosts
[INFO] Redis hostname not found in /etc/hosts
[INFO] Container IP found, adding a new record in /etc/hosts
[INFO] Search for SSL certificates generated by Traefik
[INFO] acme.json found with Traefik v2 format, dumping into pem files
[ERROR] The certificate for mail.xxx.xx or the private key was not found !
[INFO] Don't forget to add a new traefik frontend rule to generate a certificate for mail.xxx.xx subdomain
[INFO] Look /mnt/docker/traefik/acme/dump.log and 'docker logs traefik' for more information
[INFO] Starting services

dump.log

[INFO] acme.json found with Traefik v2 format, dumping into pem files
Could not read private key from <stdin>
40E79918CB7F0000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input type: PEM

Configuration (docker-compose.yml, traefik.toml...etc)

docker-compose.yml

Mailserver:
    ....
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=http_network"
      - "traefik.http.routers.spam.entrypoints=websecure"
      - "traefik.http.routers.spam.rule=Host(`spam.${MAILSERVER_DOMAIN}`)"
      - "traefik.http.routers.spam.service=spam"
      - "traefik.http.routers.spam.tls=true"
      - "traefik.http.routers.spam.tls.certresolver=letsencrypt"
      - "traefik.http.routers.spam.tls.domains[0].main=xxx.xx"
      - "traefik.http.routers.spam.tls.domains[0].sans=*.xxx.xx"

[AndrewSav]

Thank you for reporting. How can I reproduce it?

[petrleocompel]

I actually just updated traefik to version 2.10. After renewing of the certs this happened. I validated that I can extract them with traefik-certs-dumper (works).

I have no clue where to start how to simulate that. If you would lead me where this output is coming from "Could not read private key from" and where it is executed I will try to get more info why it is happening.

[AndrewSav]

Most likely it's this line: https://github.com/mailserver2/mailserver/blob/e10532a2261ae10d87dae4fbc36232538e005a03/rootfs/usr/local/bin/dumpcerts.traefik.v2.sh#L155

[petrleocompel]

Thanks I will try in few days. I will investigate and post results.

[ksylvan]

@petrleocompel Do you have any updates?

[petrleocompel]

I will have to try tomorrow. Also certs should be renewing these days. Lets see if DNS challenge got back to normal and can be parsed.

[ksylvan]

@petrleocompel Any updates on this?

[petrleocompel]

Sorry. Yes. So It few days ago it was looking like it still has issues. Currently I seems to be working fine. No clue why this happened. (There was no internet/power/system outage) Just started working and since then I have not done anything. I will try to investigate in anycase I will reopen this issue. But for now I have no info to provide sadly. Hopefully it was just signle time problem.