`/dev/stdout` is inaccessible by an unprivileged user in a container started as root

[kbrwn]

Description

Permission error received when attempting to write to /dev/stdout as a non-root user:

Steps to reproduce the issue:

# docker run -it ubuntu bash
root@2297999b6c85:/# useradd test
root@2297999b6c85:/# su test
test@2297999b6c85:/$ echo 'abc' > /dev/stdout
bash: /dev/stdout: Permission denied
test@2297999b6c85:/$ ls -la /dev/stdout
lrwxrwxrwx. 1 root root 15 Feb 21 23:20 /dev/stdout -> /proc/self/fd/1

Describe the results you received:

A permissions error is received despite the permissions being lrwxrwxrwx:

bash: /dev/stdout: Permission denied

Describe the results you expected:

echo command to /dev/stdout completes without permissions error.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-common-1.12.6-5.git037a2f5.fc25.x86_64
 Go version:      go1.7.4
 Git commit:      037a2f5/1.12.6
 Built:           Wed Jan 18 12:11:29 2017
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-common-1.12.6-5.git037a2f5.fc25.x86_64
 Go version:      go1.7.4
 Git commit:      037a2f5/1.12.6
 Built:           Wed Jan 18 12:11:29 2017
 OS/Arch:         linux/amd64

Output of docker info:

Containers: 21
 Running: 0
 Paused: 0
 Stopped: 21
Images: 11
Server Version: 1.12.6
Storage Driver: devicemapper
 Pool Name: atomicos-docker--pool
 Pool Blocksize: 524.3 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: 
 Metadata file: 
 Data Space Used: 1.285 GB
 Data Space Total: 15.77 GB
 Data Space Available: 14.49 GB
 Metadata Space Used: 843.8 kB
 Metadata Space Total: 46.14 MB
 Metadata Space Available: 45.29 MB
 Thin Pool Minimum Free Space: 1.577 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: true
 Deferred Deletion Enabled: true
 Deferred Deleted Device Count: 0
 Library Version: 1.02.136 (2016-11-05)
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: overlay null host bridge
Swarm: inactive
Runtimes: oci runc
Default Runtime: oci
Security Options: seccomp selinux
Kernel Version: 4.9.6-200.fc25.x86_64
Operating System: Fedora 25 (Atomic Host)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 1
Total Memory: 488.8 MiB
Name: localhost.localdomain
ID: QAKA:MKE4:LWLQ:O75Y:SS3B:WA33:RTUK:2Z2G:XV52:FSOB:J3HG:OTKO
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8
Registries: docker.io (secure)

Additional environment details (AWS, VirtualBox, physical, etc.):

Same result on virutualbox/vagrant (Fedora 25) and Container Linux 1325.0.0 (docker 1.13.1) Previously (1.10 and 1.11) this could be worked around by chmoding /dev/stdout.

[kbrwn]

related: https://github.com/docker/docker/issues/31106#issuecomment-281386607

[thaJeztah]

Is this related to https://github.com/docker/docker/issues/11462 ?

[bryanlatten]

Closer to (closed but not actually solved): https://github.com/docker/docker/issues/6880

[bryanlatten]

@thaJeztah this problem affects most versions of docker (latest included). The chmod workaround in https://github.com/docker/docker/issues/31106 has regressed in 1.12+, so now this is blocking a path forward for my team(s)

[bryanlatten]

@crosbymichael @mlaventure any more information I can provide? happy to work through this together

[andyneff]

Still having this problem on 17.0.5 I have to do this instead

Observation

I just noticed that this is normally how linux operates. (Running on host, no docker at all)

user1@turing ~ $ sudo su - user2
user2@turing ~ $ ls > /dev/stderr
-su: /dev/stderr: Permission denied
user2@turing ~ $ ls /dev/pts/16 -la
crw------- 1 user1 tty 136, 16 May 30 17:11 /dev/pts/16

So... do we want to close this and say "Well that's just how Linux works?" or try and overcome this, so that docker is more useable/user friendly?

---

Failing on docker 17.0.5

docker run -it --rm debian:8 bash -c "useradd test; su - test"
No directory, logging in with HOME=/
$ ls > /dev/stderr
-su: 1: cannot create /dev/stderr: Permission denied
$ bash -c "ls > /dev/stderr"
bash: /dev/stderr: Permission denied
$ bash 
test@9390f87104a5:/$ ls > /dev/stderr
bash: /dev/stderr: Permission denied
test@9390f87104a5:/$ ls /proc/$$/fd -la
total 0
dr-x------ 2 test test  0 May 30 20:51 .
dr-xr-xr-x 9 test test  0 May 30 20:51 ..
lrwx------ 1 test test 64 May 30 20:51 0 -> /13
lrwx------ 1 test test 64 May 30 20:51 1 -> /13
lrwx------ 1 test test 64 May 30 20:51 2 -> /13
lrwx------ 1 test test 64 May 30 20:51 255 -> /13

But this does work

docker exec -it 9390f87104a5 bash
test@9390f87104a5 :/$ ls > /dev/stderr
bin   dev  home  lib64  mnt  proc  run   srv  tmp  var
boot  etc  lib   media  opt  root  sbin  sys  usr

More info

docker exec -it 9390f87104a5 ls /proc/1/fd -la
total 0
dr-x------ 2 root root  0 May 30 20:57 .
dr-xr-xr-x 9 root root  0 May 30 20:57 ..
lrwx------ 1 root root 64 May 30 20:57 0 -> /13
lrwx------ 1 root root 64 May 30 20:57 1 -> /13
lrwx------ 1 root root 64 May 30 20:57 2 -> /13
lrwx------ 1 root root 64 May 30 21:01 255 -> /13

---

Workaround

But this does work

docker run -it --rm --user games debian:8 bash -c "ls > /dev/stderr"
bin   dev  home  lib64  mnt  proc  run   srv  tmp  var
boot  etc  lib   media  opt  root  sbin  sys  usr

docker run -it --rm --user games debian:8 bash
games@34d7f39fa933:/$ ls /proc/1/fd -la
total 0
dr-x------ 2 games games  0 May 30 20:54 .
dr-xr-xr-x 9 games games  0 May 30 20:54 ..
lrwx------ 1 games games 64 May 30 20:54 0 -> /13
lrwx------ 1 games games 64 May 30 20:54 1 -> /13
lrwx------ 1 games games 64 May 30 20:54 2 -> /13
lrwx------ 1 games games 64 May 30 20:54 255 -> /13

The current workaround I still have to use is:

docker run -it --rm debian:8 
root@eecfdfdd7ed5:/# useradd test
root@eecfdfdd7ed5:/# su - test
test@eecfdfdd7ed5:/tmp$ cd /tmp
test@eecfdfdd7ed5:/tmp$ touch 1
test@eecfdfdd7ed5:/tmp$ touch 12
test@eecfdfdd7ed5:/tmp$ touch 3
test@eecfdfdd7ed5:/tmp$ (ls > >(cat 1>&2)) 2> /dev/null # As you can see, no output. It's working!
test@eecfdfdd7ed5:/tmp$ (ls > >(cat 1>&2)) 1> /dev/null
test@eecfdfdd7ed5:/tmp$ 1
12
3
blah.txt

[bryanlatten]

thanks @andyneff - I'll give it a shot

[andre-richter]

I also hit this today. I am circumventing by doing a one-time

sudo chmod o+w /dev/stdout

from the user account. Any news on an official fix?

[lechup]

Thanks @andre-richter ! Do You know why this actually works? I do not see any difference in permissions after running this chmod...

I'm using this chmod o+w /dev/stdout thing as last line before exec in my custom docker-entrypoint.sh:

# beign able to write to /dev/stdout not only by root user
# https://github.com/moby/moby/issues/31243
chmod o+w /dev/stdout

exec sudo -Eu www-data "$@" >/dev/stdout 2>/dev/stdout

[andyneff]

@lechup, you are probably looking at /dev/stdout when you say the permissions didn't change. But /dev/stdout points to /proc/self/fd/1 which usually points to /dev/pts/0 or similar, and that's what you are changing permissions on.

Normally in linux, only root:tty can write directly to this.

Edit: When running a container without TTY, /proc/self/fd/0,1,2 is a pipe, with root:root permissions by default.

[javabrett]

@andyneff great tip ... an effective workaround to this issue is to elevate the non-root user to be in the tty group: better than running as root.

useradd -G tty test

Edit: Note that for this to work, you must allocate a TTY, i.e. -t or compose tty: true.

[andyneff]

Just to rehash, there three solutions to this problem

TTY

1. chmod/chown /dev/std* so that your non-root user has permission.

   • However, even if this is set in the entrypoint, when you run docker exec -t, the new pts will need to be fixed too. So that's one downside to this solution.

2. Add the tty group permissions to the user like @javabrett suggested. This is a more permanent solution, as additional docker exec -t will work.

   usermod -a -G tty test

3. A third idea involves the source of the tty permission. The group permission of pts' are controlled by the /dev/pts mount

   devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)

   • And I think (guessing) this default behavior comes from here. And currently only devices like /dev/shm have customizable behavior here
   • So if someone made a PR to add a --pts-gid flag or --pts-mode or similar, then there would be a third solution
   • I doubt this is worth it, given how easy it is to add tty perimission

So it looks like currently, 2 is the best work around

Non-TTY

• None of these solutions work.
• Instead of /dev/pts, /proc/self/fd 0-2 are pipes that you can't chmod or chown, and have no group permissions allowed. Again, this is how linux works, and is not technically a docker bug.
• I don't how the permissions of these pipes are derived, or where a workaround similar to option 3 could be added.

[nettybun]

The comments in this thread focus on accessing /dev/stdout from a shell inside the container, but I've noticed it happens outside as well.

Super standard vaultwarden that uses sqlite:

version: "3.7"
services:
  vaultwarden:
    image: vaultwarden/server:latest
    container_name: vaultwarden
    volumes:
      - ./vaultwarden-data:/data

Now from my shell not in a container I can see the files here:

nat@natbox ~> docker-compose --file vaultwarden.docker-compose.yml up -d
nat@natbox ~> ll ~/vaultwarden*
-rw-r--r-- 1 nat  nat   415 Nov 12 01:58 vaultwarden.docker-compose.yml
drwxr-xr-x 3 nat  nat  4.0K Nov 17 20:22 vaultwarden-data/                 # CONTAINER VOLUME MOUNT POINT
-rw-r--r-- 1 root root 172K Nov 12 01:58 vaultwarden-data/db.sqlite3       # INSIDE CONTAINER
-rw-r--r-- 1 root root  32K Nov 17 20:49 vaultwarden-data/db.sqlite3-shm
-rw-r--r-- 1 root root    0 Nov 17 20:22 vaultwarden-data/db.sqlite3-wal

Ok let's take a SQLite backup:

nat@natbox ~> sqlite3 ./vaultwarden-data/db.sqlite3 ".backup /dev/stdout" | restic backup --stdin --stdin-filename vault.sqlite3
Error: cannot open "/dev/stdout"
...
error: read /vault.sqlite3: no data read
Fatal: unable to save snapshot: snapshot is empty

So... I'm trying to wrap my head around this... sqlite3 is a process running on the host, as user nat, writing to /dev/stdout, and this fails? This is not running in a container (although the database is also being used by a Rust process in Vaultwarden)

For reasons I don't understand, sudo doesn't help.

[andyneff]

@heyheyhello Unfortunately your sqlite3 issue is unrelated. I don't think you use .backup that way, despite the error appearing to be a permission issue, it is not. It's an sqlite3 issue.

[speller]

As a workaround, use redirections and don't use the /dev/stdout file name. It's possible to use output redirections to run a script under a non-privileged account and redirect output to Docker logs:

$ su user -c "command" > /proc/1/fd/1 2>&1 # run command as a non-privileged user and wait for it, redirecting its output to Docker

In this example, the command's /proc/self/fd/1,2 will be accessible to open and write, so the standard outputs will be redirected to Docker logs.

To be able to run child processes in the background as non-privileged users and still log to Docker logs:

1. Add custom file descriptors to the intermediary script:

   $ su user -c "command" > /proc/1/fd/1 2>/proc/1/fd/2 3>&1 4>&2 # run the command as a non-privileged user with custom file descriptors and wait for it.  Redirect the output of the command to Docker. Redirect custom file descriptors to Docker as well.

2. Spawn background processes in the command script:

   $ sub-command >&3 2>&4 & # run background process as a non-privileged user, not wait for it, redirect its output to custom file descriptors and thus to Docker.

In this example, the "command" will spawn "sub-command" in the background which will not require any further modifications or redirections. All its standard outputs will be redirected to the custom descriptors 3 and 4 and then to Docker logs even if the parent process exited.

You may run all of this from a tty console from under the root user as well. The only limitation is that you must run everything as the root user when using docker exec and go through sus and redirections when you need to run anything as a non-privileged user and store its output in Docker logs.

For tty consoles, running su in this way:

su test > /proc/1/fd/1 2>&1

will lead to outputting everything to Docker logs and to the console simultaneously. Very similar to the subject's expected result but without using the /dev/stdout name.

I've come to this solution and it works for me for now. My task was in redirecting background non-privileged processes' output to Docker logs while the Docker entrypoint is required to be executed under the root user. While it adds workarounds, they're simple and don't require additional tools installation or fine-tuning/customizing of Docker or Linux.

[MedUnes]

a work around I found is the following:

yq -P '.' your_file.json -oy | grep '' > your_file.yaml