Cannot get new SSL certificates on Debian 12 with rootless Docker

[TailoredITRob]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes (Possible duplicates listed)

Describe the bug I have a vanilla installation of Debian 12 with rootless Docker and NPM installed. (I also have a container running Portainer.) When I try to enable SSL and add a new certificate, it fails. When I attempt to create a new Let'sEncrypt certificate from the SSL page, it also fails. Additionally, the Server Reachability test also fails with the following error.

There is a server found at this domain but it returned an unexpected status code 502. Is it the NPM server? Please make sure your domain points to the IP where your NPM instance is running.

Of course, NPM is running and I'm accessing NPM through the domain I'm attempting to pull an SSL cert for.

Inspecting the JS console, I find a request going out to http://my.domain.com/api/nginx/certificates/test-http?domains=["my.domain.com"] is failing with the following error.

{
    "error":
        {
            "code":403,
            "message": "Permission Denied"
        },
    "debug":
        {
            "stack":
                [
                    "PermissionError: Permission Denied",
                    "    at /app/lib/access.js:309:13",
                    "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
                    "    at async Object.testHttpsChallenge (/app/internal/certificate.js:1131:3)"
                ],
            "previous":
                {
                    "name": "PermissionError",
                    "message": "Permission Denied",
                    "public": true,
                    "status": 403,
                    "permission": "certificates:list"
                }
        }
}

Attempting to request a new certificate results in a 503 error from http://my.domain.com/api/nginx/certificates with the following message.

CommandError: Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

The indicated log file is never created.

Nginx Proxy Manager Version 2.11.2

To Reproduce Steps to reproduce the behavior:

1. Perform a clean install of Debian 12.
2. Install docker-ce (rootless).
3. Use the docker-compose.yml file shown below.
4. Log into NPM with default credentials, create a new password.
5. Add a new host for the any domain and attempt to add a new SSL certificate or run a reachability test.

Expected behavior A reachability test should be completed or a new Let'sEncrypt certificate issued without errors.

Operating System Debian GNU/Linux 12 (bookworm)

Additional context

services:
  portainer:
    image: 'portainer/portainer-ce:latest'
    restart: unless-stopped
    ports:
      - '127.0.0.1:9443:9443'
    volumes:
      - './storage/data:/data'
      - '/etc/localtime:/etc/localtime:ro'
      # - '/var/run/docker.sock:/var/run/docker.sock'
      - '$XDG_RUNTIME_DIR/docker.sock:/var/run/docker.sock'

  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      - '81:81' # Admin Web Port
    environment:
      # Mysql/Maria connection parameters:
      DB_MYSQL_HOST: "db"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "password"
      DB_MYSQL_NAME: "npm"
      # Uncomment this if IPv6 is not enabled on your host
      DISABLE_IPV6: 'true'
    volumes:
      - './storage/data:/data'
      - './storage/letsencrypt:/etc/letsencrypt'
    depends_on:
      - db

  db:
    image: 'jc21/mariadb-aria:latest'
    restart: unless-stopped
    environment:
      MYSQL_ROOT_PASSWORD: 'rootpassword'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'password'
      MARIADB_AUTO_UPGRADE: '1'
    volumes:
      - './storage/mysql:/var/lib/mysql'

Possibly related to #2439, #2593, and #2713.

[TailoredITRob]

I finally have this resolved.

I needed to apply the fix from #3121 as shown below.

Create a force-ssl.conf file in the same directory as docker-compose.yml and then add it to your container.

force-ssl.conf

set $test "";
if ($scheme = "http") {
    set $test "H";
}
if ($request_uri ~ "^\/\.well-known\/acme-challenge\/(.*)") {
    set $test "${test}T";
}
if ($test = H) {
    return 301 https://$host$request_uri;
}

docker-compose.yml

services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      # These ports are in format <host-port>:<container-port>
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      - '81:81' # Admin Web Port
      # Add any other Stream port you want to expose
      # - '21:21' # FTP

    # Uncomment the next line if you uncomment anything in the section
    environment:
      # Uncomment this if you want to change the location of 
      # the SQLite DB file within the container
      # DB_SQLITE_FILE: "/data/database.sqlite"

      # Uncomment this if IPv6 is not enabled on your host
      DISABLE_IPV6: 'true'

    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
      - ./force-ssl.conf:/etc/nginx/conf.d/include/force-ssl.conf   # <~~~~~~~~~~~~~~~

This should resolve the issue until PR #3121 is merged. With this I was getting back Let's Encrypt responses, but they were being denied. After inspecting the logs I realized it was failing a CAA check. (I've used other certificate providers for other subdomains. CAA records limit which CAs may issue certificates for a domain. You may need to add a CAA record for Let's Encrypt.

example.org         CAA 0 issue "letsencrypt.org"         3600

You can learn more about CAA records here.