Can't access Docker secrets when setting PUID/PGID

jonasgeiler commented 3 months ago

Version

v4.38.9

Deployment Method

Docker

Reverse Proxy

Traefik

Reverse Proxy Version

3.0.3

Description

When I set the PUID and PGID environment variables to something other than 0 for the Authelia Docker container, I get a file permission error when Authelia tries to read secrets from the secret files in /run/secrets/. The /run/secrets/ folder is owned by root, while Authelia is running under whatever I set PUID/PGID to, so it can't read those files.

A possible fix could be adding the following line to the entrypoint.sh file:

chown -R ${PUID}:${PGID} /run/secrets

I should also note that while you can set uid and gid when specifying the secrets to mount in the Docker compose file, I get the following warnings from Docker compose:

WARN[0000] secrets `uid`, `gid` and `mode` are not supported, they will be ignored 
WARN[0000] secrets `uid`, `gid` and `mode` are not supported, they will be ignored 
WARN[0000] secrets `uid`, `gid` and `mode` are not supported, they will be ignored

So it seems those are only supported in Docker Swarm.

Reproduction

Use a Docker compose file like the following (taken from docs):

---
secrets:
  JWT_SECRET:
    file: '${PWD}/data/authelia/secrets/JWT_SECRET'
  SESSION_SECRET:
    file: '${PWD}/data/authelia/secrets/SESSION_SECRET'
  STORAGE_PASSWORD:
    file: '${PWD}/data/authelia/secrets/STORAGE_PASSWORD'
  STORAGE_ENCRYPTION_KEY:
    file: '${PWD}/data/authelia/secrets/STORAGE_ENCRYPTION_KEY'
services:
  authelia:
    container_name: 'authelia'
    image: 'docker.io/authelia/authelia:latest'
    restart: 'unless-stopped'
    networks:
      net:
        aliases: []
    expose:
      - 9091
    secrets: ['JWT_SECRET', 'SESSION_SECRET', 'STORAGE_PASSWORD', 'STORAGE_ENCRYPTION_KEY']
    environment:
      AUTHELIA_IDENTITY_VALIDATION_RESET_PASSWORD_JWT_SECRET_FILE: '/run/secrets/JWT_SECRET'
      AUTHELIA_SESSION_SECRET_FILE: '/run/secrets/SESSION_SECRET'
      AUTHELIA_STORAGE_POSTGRES_PASSWORD_FILE: '/run/secrets/STORAGE_PASSWORD'
      AUTHELIA_STORAGE_ENCRYPTION_KEY_FILE: '/run/secrets/STORAGE_ENCRYPTION_KEY'
      # ---
      # THIS IS THE IMPORTANT PART:
      PUID: 1000
      PGID: 1000
      # ---
    volumes:
      - '${PWD}/data/authelia/config:/config'
networks:
  net:
    external: true
    name: 'net'
...

Expectations

Authelia can read secrets from the Docker secrets files.

Configuration (Authelia)

Set in Docker compose as environment variables:

TZ: Europe/vienna
PUID: 1000
PGID: 1000

# access control
AUTHELIA_ACCESS_CONTROL_DEFAULT_POLICY: two_factor

# password policy
AUTHELIA_PASSWORD_POLICY_ZXCVBN_ENABLED: true
AUTHELIA_PASSWORD_POLICY_ZXCVBN_MIN_SCORE: 4

# identity validation
AUTHELIA_IDENTITY_VALIDATION_RESET_PASSWORD_JWT_SECRET_FILE: /run/secrets/authelia_jwt_secret

# session
AUTHELIA_SESSION_SECRET_FILE: /run/secrets/authelia_session_secret
AUTHELIA_SESSION_REDIS_HOST: authelia-redis
AUTHELIA_SESSION_REDIS_PASSWORD: authelia

# storage
AUTHELIA_STORAGE_ENCRYPTION_KEY_FILE: /run/secrets/authelia_storage_encryption_key
AUTHELIA_STORAGE_LOCAL_PATH: /config/db.sqlite3

# notifications
AUTHELIA_NOTIFIER_SMTP_ADDRESS: *smtp_address
AUTHELIA_NOTIFIER_SMTP_USERNAME: *smtp_username
AUTHELIA_NOTIFIER_SMTP_PASSWORD: *smtp_password
AUTHELIA_NOTIFIER_SMTP_SENDER: *authelia_smtp_sender
AUTHELIA_NOTIFIER_SMTP_IDENTIFIER: *authelia_domain
AUTHELIA_NOTIFIER_SMTP_SUBJECT: '{title}'
AUTHELIA_NOTIFIER_SMTP_STARTUP_CHECK_ADDRESS: *smtp_from

# miscellaneous
AUTHELIA_THEME: auto

# custom
CUSTOM_DOMAIN: *root_domain
CUSTOM_AUTHELIA_URL: *authelia_url

Set in the configuration.yaml:

session:
  cookies:
    - domain: '{{ mustEnv "CUSTOM_DOMAIN" }}'
      authelia_url: '{{ mustEnv "CUSTOM_AUTHELIA_URL" }}'

Build Information

Last Tag: v4.38.9
State: tagged clean
Branch: v4.38.9
Commit: 2798576ee25a56fd4c14814acd087b6e92f3978b
Build Number: 30034
Build OS: linux
Build Arch: amd64
Build Compiler: gc
Build Date: Sun, 16 Jun 2024 19:47:16 +1000
Extra: 

Go:
    Version: go1.22.2
    Module Path: github.com/authelia/authelia/v4
    Executable Path: github.com/authelia/authelia/v4/cmd/authelia

Logs (Authelia)

authelia-1  | time="2024-06-30T08:35:18+02:00" level=warning msg="Configuration: access_control: no rules have been specified so the 'default_policy' of 'two_factor' is going to be applied to all requests"
authelia-1  | time="2024-06-30T08:35:18+02:00" level=error msg="Configuration: secrets: error loading secret path /run/secrets/authelia_jwt_secret into key 'identity_validation.reset_password.jwt_secret': file permission error occurred: open /run/secrets/authelia_jwt_secret: permission denied"
authelia-1  | time="2024-06-30T08:35:18+02:00" level=error msg="Configuration: secrets: error loading secret path /run/secrets/authelia_session_secret into key 'session.secret': file permission error occurred: open /run/secrets/authelia_session_secret: permission denied"
authelia-1  | time="2024-06-30T08:35:18+02:00" level=error msg="Configuration: secrets: error loading secret path /run/secrets/authelia_storage_encryption_key into key 'storage.encryption_key': file permission error occurred: open /run/secrets/authelia_storage_encryption_key: permission denied"
authelia-1  | time="2024-06-30T08:35:18+02:00" level=error msg="Configuration: session: option 'secret' is required when using the 'redis' provider"
authelia-1  | time="2024-06-30T08:35:18+02:00" level=error msg="Configuration: storage: option 'encryption_key' is required"
authelia-1  | time="2024-06-30T08:35:18+02:00" level=error msg="Configuration: identity_validation: reset_password: option 'jwt_secret' is required when the reset password functionality isn't disabled"
authelia-1  | time="2024-06-30T08:35:18+02:00" level=fatal msg="Can't continue due to the errors loading the configuration" stack="github.com/authelia/authelia/v4/internal/commands/context.go:327 (*CmdCtx).ConfigValidateLogRunE\ngithub.com/authelia/authelia/v4/internal/commands/context.go:186 NewRootCmd.(*CmdCtx).ChainRunE.func1\ngithub.com/spf13/cobra@v1.8.1/command.go:970                     (*Command).execute\ngithub.com/spf13/cobra@v1.8.1/command.go:1117                    (*Command).ExecuteC\ngithub.com/spf13/cobra@v1.8.1/command.go:1041                    (*Command).Execute\ngithub.com/authelia/authelia/v4/cmd/authelia/main.go:10          main\nruntime/internal/atomic/types.go:194                             (*Uint32).Load\nruntime/asm_amd64.s:1695                                         goexit"

Logs (Proxy / Application)

N/A

Documentation

No response

Pre-Submission Checklist

[X] I agree to follow the Code of Conduct
[X] This is a bug report and not a support request
[X] I have read the security policy and this bug report is not a security issue or security related issue
[X] I have either included the complete configuration file or I am sure it's unrelated to the configuration
[X] I have either included the complete debug / trace logs or the output of the build-info command if the logs are not relevant
[X] I have provided all of the required information in full with the only alteration being reasonable sanitization in accordance with the Troubleshooting Sanitization reference guide
[X] I have checked for related proxy or application logs and included them if available
[X] I have checked for related issues and checked the documentation

glime-ai[bot] commented 3 months ago

Please note this response was created using generative AI leveraging previously available information from the repository. This information may be helpful however it is our strong recommendation that you carefully consider the information before blindly applying it.

The issue you're encountering is due to the file permission settings of the /run/secrets/ directory, which is owned by root. When you set the PUID and PGID environment variables to non-root values, the Authelia container cannot access these files.

To resolve this, you can indeed modify the entrypoint.sh script to change the ownership of the /run/secrets/ directory to the specified PUID and PGID. Here is the suggested line to add to the entrypoint.sh file:

chown -R ${PUID}:${PGID} /run/secrets

This will ensure that the Authelia process running under the specified PUID and PGID can read the secret files.

Alternatively, you can also set the permissions of the secret files to be readable by the group or others, but this is less secure and not recommended.

For more detailed information, you can refer to the Authelia documentation on Docker deployment and permissions:

This comment was generated by Glime.

james-d-elliott commented 3 months ago

Use docker to specify the correct permissions, this is working as intended as the files mounted here are controlled by docker entirely. As this is the intended method for docker, if this is not working there is an issue on their end. Naturally we also have always recommended using the user parameters not the PUID/PGID env vars. Alternatively just mount them as a directory considering there is no benefit to mounting them in this way without swarm.

https://docs.docker.com/compose/compose-file/05-services/#long-syntax-4

jonasgeiler commented 3 months ago

@james-d-elliott wrote: Use docker to specify the correct permissions, this is working as intended as the files mounted here are controlled by docker entirely. As this is the intended method for docker, if this is not working there is an issue on their end. Naturally we also have always recommended using the user parameters not the PUID/PGID env vars. Alternatively just mount them as a directory considering there is no benefit to mounting them in this way without swarm.

https://docs.docker.com/compose/compose-file/05-services/#long-syntax-4

I've read the documentation you linked and tried it out shortly after submitting this issue. Maybe you have read the issue before I edited with the new information:

The uid and gid fields are seemingly not supported outside of Docker Swarm. I get the following warnings from Docker compose:

WARN[0000] secrets `uid`, `gid` and `mode` are not supported, they will be ignored 
WARN[0000] secrets `uid`, `gid` and `mode` are not supported, they will be ignored 
WARN[0000] secrets `uid`, `gid` and `mode` are not supported, they will be ignored

The secrets attribute for the service in the Docker compose file looks something like this:

secrets:
  - source: authelia_jwt_secret
    uid: 1000
    gid: 1000
  - source: authelia_session_secret
    uid: 1000
    gid: 1000
  - source: authelia_storage_encryption_key
    uid: 1000
    gid: 1000

And it's not working... Maybe I'll ask the Docker guys if they can add proper support for this? The problem COULD be solved by Docker itself, however IF it's possible and they decide to add it and WHEN they will do it I don't know... For this reason a temporary fix on your side would be very appreciated!

I am currently trying to get setting the user attribute for the service to work (instead of using PUID/PGID) but for some reason I get the same error... Will see if I can fix it.

james-d-elliott commented 3 months ago

Apologies but we're not crossing this barrier of responsibility as clawing back these bad features is almost impossible without people complaining. Even if we wanted to it's technically impossible as these files are part of a read-only file system. Feel free to try to chown any file mounted in /run/secrets for yourself to see what I mean.

The correct fix must come from the docker side as it's part of their domain logic. Seems like a pretty asinine for them to have the user option and secrets option when the user has only the path of using an privilege escalation vulnerability to gain access to the secrets since it never has access under uid/gid 0.

As far as how to work around it you can mount the files as part of a directory owned and only readable by the correct user id. You can choose an arbitrary number for this with the chown 1234:0 /path/to/file && chmod 600 /path/to/file command.

This has the same benefits as mounting them as secrets. The secrets syntax is almost exclusively at the benefit of docker swarm where the files may not be local.

jonasgeiler commented 3 months ago

@james-d-elliott I actually found a Docker issue mentioning this problem, and below it there was this comment: https://github.com/docker/compose/issues/9648#issuecomment-1824340917 So it looks like they will not fix it for now. Really sucks but what shall you do...

I guess I'll look for a workaround using a custom image or something. Thanks anyways!

Btw setting the user attribute for the service in the Docker compose file also didn't work, since the secrets are still mounted as root, even if authelia is running under a different user...

james-d-elliott commented 3 months ago

As I already mentioned there is no way around it. Docker forcibly mounts the /run/secrets as read-only. You can't change the permissions of these files. It seems by their implementation the only way to handle this is exactly as I described, mount file as a volume with the correct permissions.

jonasgeiler commented 3 months ago

As I already mentioned there is no way around it. Docker forcibly mounts the /run/secrets as read-only. You can't change the permissions of these files. It seems by their implementation the only way to handle this is exactly as I described, mount file as a volume with the correct permissions.

I admit I misread the part where you said it was "technically impossible", but it's still a good idea to link to the Docker issue for other people in the future in case someone stumbles across this issue. Thanks again!

james-d-elliott commented 3 months ago

Yeah I agree. not opposed to linking their issue. No worries at all, feel free to open a discussion if you want examples of the proposed solution. I find it fascinating that setting the user no longer sets the secret permissions, I know for certain it did in the past.

It's also probably relevant to link the following where it actually quite convincingly suggests secrets are only intended for swarm mode, quite literally seems like a docker compose hack to work around this limitation as it's not a feature of standard docker (first result for "docker secrets"): https://docs.docker.com/engine/swarm/secrets/

For posterity:

jonasgeiler commented 3 months ago

Hold on a second 😅

I just noticed that using secrets read from environment variables work! They are defined like this:

secrets:
  jwt_secret:
    environment: JWT_SECRET

After mounting them in the container, Authelia could read them normally. The reason is that they have a 0444 permission inside the container, set by Docker by default. The file that I used previously had 0400 permissions, which were also used inside of the container, so Authelia could not read the secret. This means that everything is actually working fine and all I had to do is set 0444 permissions on the source secret file on the host... I feel stupid haha

james-d-elliott commented 3 months ago

I suspect it's related to the way that docker compose handles that in standalone containers. Since secrets are not actually available for standalone containers it's just bind mounting them, so they should retain the permissions, i.e. owner, group, and mod. Would explain why it wasn't working now and I knew the user config worked in the past.

authelia / authelia