Volume permissions issue

[atamti]

Description:

I'm attempting to run vaultwarden and caddy in rootless docker. I had it working fine in "rootful" docker, but this gave me issues. I've worked through all of them but a stubborn permissions error, that seems to be driven by a change of directory ownership of the caddy-data volume from "dockeruser" to "nobody" during the volume mounting process. This has really stumpted both me and chatGPT (at least based on what prompts I gave it), so any help would be greatly appreciated. Happy to provide more info if helpful.

Environment:

Docker version: 24.0.4 build 3713ee1 (rootless) Vaultwarden version: 1.29.0 Caddy version: 2.6.4 Operating system: Raspberry Pi OS (Bullseye)

Configuration:

Docker-compose.yml

Note: Anything in [ ] I removed for privacy

version: '3'

services:
  vaultwarden:
    image: vaultwarden/server:1.29.0
    container_name: vaultwarden
    user: 1002:1002
    restart: unless-stopped
    environment:
      - WEBSOCKET_ENABLED=true
      - DOMAIN=[mydomain]  # Your domain; vaultwarden needs to know it's https to work properly with attachments
      - LOGIN_RATELIMIT_MAX_BURST=10
      - LOGIN_RATELIMIT_SECONDS=60
      - ADMIN_RATELIMIT_MAX_BURST=10
      - ADMIN_RATELIMIT_SECONDS=60
      - ADMIN_TOKEN=[Redacted]
      - SENDS_ALLOWED=true
      - EMERGENCY_ACCESS_ALLOWED=true
      - WEB_VAULT_ENABLED=true
      - SIGNUPS_ALLOWED=false
      - SIGNUPS_VERIFY=true
      - SIGNUPS_VERIFY_RESEND_TIME=3600
      - SIGNUPS_VERIFY_RESEND_LIMIT=5
      - SMTP_HOST=[myemail]
      - SMTP_FROM=[myemail]
      - SMTP_PORT=465
      - SMTP_SECURITY=force_tls
      - SMTP_USERNAME=[Redacted]
      - SMTP_PASSWORD=[Redacted]
    volumes:
      - ./vw-data:/data

  caddy:
    image: caddy:2.6.4
    container_name: caddy
    user: 1002:1002
    restart: unless-stopped
    ports:
      - 8080:8080  # Needed for the ACME HTTP-01 challenge.
      - 8443:8443
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - ./caddy-config:/config
      - ./caddy-data:/data
    environment:
      DOMAIN: '[my domain]'  # Your domain.
      EMAIL: '[my email]'                 # The email address to use for ACME registration.
      LOG_FILE: '/data/access.log'

Caddy configuration (e.g., Caddyfile)

{$DOMAIN}:8443 {
  log {
    level INFO
    output file {$LOG_FILE} {
      roll_size 10MB
      roll_keep 10
    }
  }

  # Use the ACME HTTP-01 challenge to get a cert for the configured domain.
  tls {$EMAIL}

  # This setting may have compatibility issues with some browsers
  # (e.g., attachment downloading on Firefox). Try disabling this
  # if you encounter issues.
  encode gzip

  # Notifications redirected to the WebSocket server
  reverse_proxy /notifications/hub vaultwarden:3012

  # Proxy everything Rocket
  reverse_proxy vaultwarden:80 {
       # Send the true remote IP to Rocket, so that vaultwarden can put this in the
       # log, so that fail2ban can ban the correct IP.
       header_up X-Real-IP {remote_host}
  }
}

Steps to reproduce:

1. Install rootless docker and create non-root user to run the docker.service daemon, in my case 'dockerdaemon'.
2. Create a separate user to run inside the container ('dockeruser' in my case). Customize USER field for the UID of your created user.
3. Take this Caddyfile and docker compose file and customize it for an available domain / email address.
4. Properly configure port forwarding in a router (443 -> 8443 and 80 -> 8080), as well as other DNS / firewall settings.
5. Start the containers (docker compose up -d) as dockerdaemon.

Expected behavior:

Docker compose file imports the volumes maintaining current user ownership and permissions (dockeruser, uid=1002)

Actual behavior:

Docker compose file imports the volumes owned by "nobody" with group "nobody" (note -not "nogroup")

Additional information

1. I can get this working completely by maintaining my existing SSL certs (obtained before I installed rootless docker) and opening up view permissions of the directory to all users (I'm the only operator of the server, but this is of course bad security practice).
2. UID/GID of the user running inside the container is indeed 1002 (dockeruser).
3. Error as listed in the caddy log: {"level":"error","ts":1689864895.27265,"logger":"tls","msg":"job failed","error":"[my site]: obtaining certificate: failed storage check: open /data/caddy/rw_test_2276075538939889706: permission denied - storage is probably misconfigured"}

Attempts made:

1. Using user namespaces: Adding a daemon.json file with "userns-remap": "default"
2. Validating that dockeruser properly owns the entire caddy-data directory
3. Deleting the directory that stores the volume and recreating / changing permissions to dockeruser
4. Appending ",uid=1002,gid=1002" to the end of each named volume (changes ownership from "nobody" to "root" but no change to the issue)
5. Adding a Dockerfile and entrypoint.sh to force change the permissions back to dockeruser (ran into a VERY stubborn '/entrypoint.sh: no such file or directory' error in the caddy container, which then refused to start)
6. Restarting docker.service
7. Repulling both images
8. Restarting the containers

[atamti]

Resolved - consolidated my users and ran as root inside the container.

Ended up moving to nginx as I use it for other things and wanted to centalize control of my ports, but I did have it working as containerized caddy as well