--publish: docker flag / tcp port not bind properly

[eine]

I am trying to execute the equivalent to the following command with x11docker:

# winpty docker run --rm -p 5555:80 ghdl/ext:ide
Listening on [::]:80

NOTE: you can browse localhost:5555 on the host.

However, with x11docker, the port seems not to be set properly:

# ./x11docker -- -p 5555:80 -- ghdl/ext:ide
./x11docker: line 5240: systemctl: command not found
x11docker note: Using X server option --vcxsrv

x11docker note: Per default x11docker stores its cache files on drive C:.
  docker setup may not allow to share files from drive C:.
  If startup fails with an 'access denied' error,
  please either allow access to drive C: or specify a custom folder for cache
  storage with option '--cachebasedir D:/some/cache/folder'.
  Same issue can occur with option '--home'.
  Use option '--homebasedir D:/some/home/folder' in that case.

x11docker note: Did not find container init system 'tini'.
  This is a bug in your distributions docker package.
  Normally, docker provides init system tini as '/usr/bin/docker-init'.

  x11docker uses tini for clean process handling and fast container shutdown.
  To provide tini yourself, please download tini-static:
    https://github.com/krallin/tini/releases/download/v0.18.0/tini-static
  Store it in one of:
    /home/eine/.local/share/x11docker/
    /usr/local/share/x11docker/

ps: unknown option -- o
Try `ps --help' for more information.
2019/01/20 03:42:13 listen tcp :80: bind: permission denied

# ./x11docker --debug -- -p 5555:80 -- ghdl/ext:ide
DEBUGNOTE[647.95]: Command at Line 6686 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6823 - ::main::main
./x11docker: line 6686: PRETTY_NAME: unbound variable
DEBUGNOTE[648.07]:
x11docker version: 5.3.4-beta
docker version:    Docker version 18.09.1, build 4c52b90
Host system:
Command:           ./x11docker '--debug' '--' '-p' '5555:80' '--' 'ghdl/ext:ide'
Parsed options:     --debug -- '-p' '5555:80' '--' 'ghdl/ext:ide'
DEBUGNOTE[648.32]: Command at Line 5082 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6694 - ::check_host::main::main
./x11docker: line 5082: ID: unbound variable
DEBUGNOTE[648.44]: Command at Line 5082 returned with error code 1:
  Hostsystem="$(source /etc/os-release 2>/dev/null; echo $ID)"
  6694 - ::check_host::main::main
DEBUGNOTE[650.57]: ps can not watch root processes.
./x11docker: line 5240: systemctl: command not found
./x11docker: line 5270: XDG_RUNTIME_DIR: unbound variable
DEBUGNOTE[653.93]: Terminating x11docker.
DEBUGNOTE[654.12]: List of stored background processes:

DEBUGNOTE[654.22]: time to say goodbye (finish)
DEBUGNOTE[654.42]: Exitcode 0

This is quite surprising, because the same command with a different container (which does not set any service in any port), seems to work properly:

# ./x11docker -- -p 5555:80 -- alpine sh
./x11docker: line 5240: systemctl: command not found
x11docker note: Using X server option --vcxsrv

x11docker note: Per default x11docker stores its cache files on drive C:.
  docker setup may not allow to share files from drive C:.
  If startup fails with an 'access denied' error,
  please either allow access to drive C: or specify a custom folder for cache
  storage with option '--cachebasedir D:/some/cache/folder'.
  Same issue can occur with option '--home'.
  Use option '--homebasedir D:/some/home/folder' in that case.

x11docker note: Did not find container init system 'tini'.
  This is a bug in your distributions docker package.
  Normally, docker provides init system tini as '/usr/bin/docker-init'.

  x11docker uses tini for clean process handling and fast container shutdown.
  To provide tini yourself, please download tini-static:
    https://github.com/krallin/tini/releases/download/v0.18.0/tini-static
  Store it in one of:
    /home/eine/.local/share/x11docker/
    /usr/local/share/x11docker/

ps: unknown option -- o
Try `ps --help' for more information.

# docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                  NAMES
f0f11a40d94d        alpine              "env /bin/sh - /x11d…"   About a minute ago   Up About a minute   0.0.0.0:5555->80/tcp   x11docker_X100_dd881c_alpine

But it fails with this one:

# ./x11docker --debug -- -p 5555:80 -- alpine sh
DEBUGNOTE[234.64]: Command at Line 6686 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6823 - ::main::main
./x11docker: line 6686: PRETTY_NAME: unbound variable
DEBUGNOTE[234.81]:
x11docker version: 5.3.4-beta
docker version:    Docker version 18.09.1, build 4c52b90
Host system:
Command:           ./x11docker '--debug' '--' '-p' '5555:80' '--' 'alpine' 'sh'
Parsed options:     --debug -- '-p' '5555:80' '--' 'alpine' 'sh'
DEBUGNOTE[235.09]: Command at Line 5082 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6694 - ::check_host::main::main
./x11docker: line 5082: ID: unbound variable
DEBUGNOTE[235.23]: Command at Line 5082 returned with error code 1:
  Hostsystem="$(source /etc/os-release 2>/dev/null; echo $ID)"
  6694 - ::check_host::main::main
DEBUGNOTE[237.52]: ps can not watch root processes.
./x11docker: line 5240: systemctl: command not found
./x11docker: line 5270: XDG_RUNTIME_DIR: unbound variable
DEBUGNOTE[241.12]: Terminating x11docker.
DEBUGNOTE[241.31]: List of stored background processes:

DEBUGNOTE[241.40]: time to say goodbye (finish)
DEBUGNOTE[241.57]: Exitcode 0

./x11docker -- -p 5555:80 -- simexp/octave bash produces the same result as ./x11docker -- -p 5555:80 -- alpine sh. Also, ./x11docker --debug -- -p 5555:80 -- simexp/octave bash produces the same result as ./x11docker --debug -- -p 5555:80 -- alpine sh

[mviereck]

As a first feedback: This one works:

x11docker --user=root --cap-default -- -p 5555:80 -- ghdl/ext:ide

Probably the container needs capability NET_BIND_SERVICE. I am not sure yet why it also needs user root. That might as well be an issue within the image. Or an unprivileged container user needs an additional group membership that could be added with x11docker option --group-add.

Edit: Some useful hints, maybe I can something include in x11docker: https://superuser.com/questions/710253/allow-non-root-process-to-bind-to-port-80-and-443/892391

---

--debug fails due to unbound variables. That is a different issue, I'll fix it soon. In debug mode x11docker terminates if an undefined variable is used.

Edit: The unbound variables and error messages of systemctl and ps -o are fixed now. Hopefully x11docker will run flawlessly now with --debug in MSYS2.

[mviereck]

Some progress: I've build a new image:

FROM ghdl/ext:ide
RUN apt-get update && apt-get install -y libcap2-bin
RUN /sbin/setcap CAP_NET_BIND_SERVICE=+eip /opt/filebrowser/filebrowser

Now it successfully binds the port. It fails now accessing a file:

$ x11docker --tty --cap-default -- -p 5555:80 -- setcapimage
Listening on [::]:80
2019/01/21 16:31:01 open /opt/filebrowser/filebrowser.db: permission denied

I'd say a file brower should be able to run without root permissions. I don't have the insight to say how it should be set up properly within this image. If files owned by the user located in HOME are needed, you can store them in /etc/skel. They will be copied to HOME on x11docker's user creation if HOME is empty.

[eine]

As a first feedback: This one works: x11docker --user=root --cap-default -- -p 5555:80 -- ghdl/ext:ide

Probably the container needs capability NET_BIND_SERVICE. I am not sure yet why it also needs user root. That might as well be an issue within the image. Or an unprivileged container user needs an additional group membership that could be added with x11docker option --group-add.

On the one hand, the image is not meant to be started with a non-root user. Hence, some content is located in paths where any user cannot access by default (e.g. /opt). On the other hand, as you found, root permissions are required in order to serve at port 80. So, I tried:

# ./x11docker -- -p 5555:8080 -- ghdl/ext:ide -p 8080
...
Listening on [::]:8080
2019/01/21 20:33:33 open /opt/filebrowser/filebrowser.db: permission denied

and, as long as the port is concerned, it works. Therefore, I think that using a different port is a good workaround, rather than fighting with additional parameters. Indeed, the default port in newer versions of filebrowser is 8080: https://github.com/filebrowser/filebrowser/blob/master/cmd/root.go#L50

--debug fails due to unbound variables. That is a different issue, I'll fix it soon. In debug mode x11docker terminates if an undefined variable is used.

Edit: The unbound variables and error messages of systemctl and ps -o are fixed now. Hopefully x11docker will run flawlessly now with --debug in MSYS2.

I think that this changes might have introduced some other bug. As commented in #87, I am having problems with --debug and/or i. Now, I tried:

# ./x11docker --user=root -- -p 5555:8080 -- ghdl/ext:ide -p 8080
x11docker note: Using X server option --vcxsrv

x11docker note: Per default x11docker stores its cache files on drive C:.
  docker setup may not allow to share files from drive C:.
  If startup fails with an 'access denied' error,
  please either allow access to drive C: or specify a custom folder for cache
  storage with option '--cachebasedir D:/some/cache/folder'.
  Same issue can occur with option '--home'.
  Use option '--homebasedir D:/some/home/folder' in that case.

x11docker ERROR: Option --user: Unknown host user or invalid user number 'root'.
  Non-host users can be specified with an UID only, not with a name.

  Type 'x11docker --help' for usage information
  Debug options: '--verbose' (full log) or '--debug' (log excerpt).
  Logfile will be: /c/Users/eine/x11docker/cache/x11docker.log
  Please report issues at https://github.com/mviereck/x11docker

# ./x11docker --debug --user=root -- -p 5555:8080 -- ghdl/ext:ide -p 8080
DEBUGNOTE[529.03]: Command at Line 6693 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6830 - ::main::main
DEBUGNOTE[529.14]:
x11docker version: 5.3.4-beta
docker version:    Docker version 18.09.1, build 4c52b90
Host system:
Command:           ./x11docker '--debug' '--user=root' '--' '-p' '5555:8080' '--' 'ghdl/ext:ide' '-p' '8080'
Parsed options:     --debug --user 'root' -- '-p' '5555:8080' '--' 'ghdl/ext:ide' '-p' '8080'
DEBUGNOTE[529.32]: Command at Line 5088 returned with error code 1:
  source /etc/os-release 2> /dev/null
  6701 - ::check_host::main::main
DEBUGNOTE[535.26]: Dependency check for --vcxsrv: 0
DEBUGNOTE[535.75]: Dependency check for --vcxsrv: 0
DEBUGNOTE[536.26]: Dependency check for --vcxsrv: 0
x11docker note: Using X server option --vcxsrv

x11docker note: Per default x11docker stores its cache files on drive C:.
  docker setup may not allow to share files from drive C:.
  If startup fails with an 'access denied' error,
  please either allow access to drive C: or specify a custom folder for cache
  storage with option '--cachebasedir D:/some/cache/folder'.
  Same issue can occur with option '--home'.
  Use option '--homebasedir D:/some/home/folder' in that case.

x11docker ERROR: Option --user: Unknown host user or invalid user number 'root'.
  Non-host users can be specified with an UID only, not with a name.

  Type 'x11docker --help' for usage information
  Debug options: '--verbose' (full log) or '--debug' (log excerpt).
  Logfile will be: /c/Users/eine/x11docker/cache/x11docker.log
  Please report issues at https://github.com/mviereck/x11docker

DEBUGNOTE[536.87]: Command at Line 409 returned with error code 1:
  return 1
  3380 - ::error::check_containeruser::main::main
DEBUGNOTE[536.98]: Terminating x11docker.
DEBUGNOTE[537.17]: List of stored background processes:

DEBUGNOTE[537.28]: time to say goodbye (finish)
DEBUGNOTE[537.44]: Exitcode 1

[eine]

Some progress: ... Now it successfully binds the port. It fails now accessing a file:

This is cool. As I said above, I don't think it is the best solution for this use case, but I'll keep it as a reference.

I'd say a file brower should be able to run without root permissions. I don't have the insight to say how it should be set up properly within this image. If files owned by the user located in HOME are needed, you can store them in /etc/skel. They will be copied to HOME on x11docker's user creation if HOME is empty.

Agree. I am going to rework the image accordingly.

File Browser needs acces to two paths. The first one is /opt/filebrowser, where filebrowser (binary), .filebrowser.yml and filebrowser.db are located. Only the latter must be writable. The second path, /srv by default, is where data/content to be served is saved.

Do you think that paths should default to ~/.filebrowser/filebrowser.db and ~/.filebrowser/srv? Or to ~/skel/filebrowser.db and ~/skel/srv?

[mviereck]

On the one hand, the image is not meant to be started with a non-root user.

x11docker ERROR: Option --user: Unknown host user or invalid user number 'root'.

oh, ok, MSYS2 does not have a user root. x11docker could handle that. Instead of --user=root try --user=0. If that fails, too, try --user=RETAIN.

I think that this changes might have introduced some other bug. As commented in #87, I am having problems with --debug and/or i.

This changes cannot be the source of the issue. I suspect an issue in docker itself. I am still looking at your --debug outputs in #87. A restart of docker daemon (or entire Windows) may already help.

Do you think that paths should default to ~/.filebrowser/filebrowser.db and ~/.filebrowser/srv? Or to ~/skel/filebrowser.db and ~/skel/srv?

They should default to HOME. /etc/skel is a folder to create a skeleton file system that is provided as content of HOME for every new user. There one can e.g. place empty folders like Documents, Videos, and prepared configuration files that every new user should get. It is not meant for dynamic content.

Instead of ~/.filebrowser I'd recommend ~/.local/share/filebrowser. Possible config files should be in ~/.config/filebrowser, and cache files in ~/.cache/filebrowser. This is a new standard defined somewhere to decrease the flood of dotfiles in HOME.

The second path, /srv by default, is where data/content to be served is saved. ~/.filebrowser/srv

I'd say content should not be hidden. Maybe placing it in ~/filebrowser without a dot? If it can be hidden, store it in ~/.local/share/filebrowser or ~/.local/share/filebrowser/srv.

On regular Linux systems, x11docker stores files of --home in ~/.local/share/x11docker and soft-links it to ~/x11docker.

[eine]

oh, ok, MSYS2 does not have a user root. x11docker could handle that. Instead of --user=root try --user=0. If that fails, too, try --user=RETAIN.

./x11docker --user=0 -- -p 5555:8080 -- ghdl/ext:ide -p 8080 does work. ./x11docker --user=RETAIN -- -p 5555:8080 -- ghdl/ext:ide -p 8080 too.

But, with both of them, when I Ctrl+C to terminate, the log seems correct, the X server is closed, but the container is not:

Listening on [::]:8080
x11docker note: User in container: uid=0(root) gid=0(root) groups=0(root)
root:x:0:0:root:/root:/bin/bash

DEBUGNOTE[151.43]: Received SIGINT
DEBUGNOTE[151.53]: Terminating x11docker.
DEBUGNOTE[151.72]: List of stored background processes:
15452 tailstdout
2600 tailstderr
3300 watchpidlist
10236 watchmessagefifo
8056 containershell
10844 Xserver
DEBUGNOTE[152.01]: Checking: 10844 (Xserver):
DEBUGNOTE[152.30]: Checking: 8056 (containershell):
DEBUGNOTE[152.58]: Checking: 10236 (watchmessagefifo):
DEBUGNOTE[152.86]: Checking: 3300 (watchpidlist):
DEBUGNOTE[153.16]: Checking: 2600 (tailstderr):
DEBUGNOTE[153.48]: Checking: 15452 (tailstdout):
DEBUGNOTE[153.60]: time to say goodbye (finish)
DEBUGNOTE[153.78]: Exitcode 0

I need to docker rm -f x11docker_X100_....

A restart of docker daemon (or entire Windows) may already help.

I did restart it a couple of times. Just in case.

They should default to HOME. /etc/skel is a folder to create a skeleton file system that is provided as content of HOME for every new user. There one can e.g. place empty folders like Documents, Videos, and prepared configuration files that every new user should get. It is not meant for dynamic content.

Got it. Thanks.

Instead of ~/.filebrowser I'd recommend ~/.local/share/filebrowser. Possible config files should be in ~/.config/filebrowser, and cache files in ~/.cache/filebrowser. This is a new standard defined somewhere to decrease the flood of dotfiles in HOME.

I'd say content should not be hidden. Maybe placing it in ~/filebrowser without a dot? If it can be hidden, store it in ~/.local/share/filebrowser or ~/.local/share/filebrowser/srv.

On regular Linux systems, x11docker stores files of --home in ~/.local/share/x11docker and soft-links it to ~/x11docker.

Sure. Thanks again.

---

I'm closing this issue, because the problem with ports is solved.

[mviereck]

But, with both of them, when I Ctrl+C to terminate, the log seems correct, the X server is closed, but the container is not:

I'll look at it in #106.

[drzraf]

Trying Chrome + debugging websocket at port 9222: $ x11docker --cap-default -g --nxagent --no-entrypoint -- -p 9222:9222 -- zenika/alpine-chrome:with-webgl chromium-browser --no-sandbox --remote-debugging-address=0.0.0.0 --remote-debugging-port=9222

Sadly, ws connection does not work, contrary to the bare run (not using the webgl build image which does not run natively): $ docker container run -it -p 9222:9222 --rm --entrypoint '' zenika/alpine-chrome:86 chromium-browser --no-sandbox --remote-debugging-address=0.0.0.0 --remote-debugging-port=9222 --headless

From the two docker inspect I couldn't spot what's wrong.

• docker-inspect-native-alpine-chrome-86.txt
• x11docker-inspect-alpine-chrome-webgl.txt

[mviereck]

@drzraf I've opened a new ticket #332, let's continue there.