docker-for-win: General issue tracker: MSYS2/Cygwin/WSL, --vcxsrv, --xwin

[mviereck]

Coming from #55:

---

I found an interesting comment in this docker-WSL-HowTo. You would only need to create a bash alias that translates command docker to /mnt/c/xyz/docker.exe. Maybe it even works ootb without an alias if docker is not installed in WSL and docker.exe is within PATH:

There is a simpler way. Install docker for windows, then WSL/Ubuntu and in your bash profile create an alias for docker=path-to-docker.exe In WSL you can execute windows commands. This way you are in the the Ubuntu Bash shell but executing the windows docker command. No mapping, no docker client install.

It seems to be unusable for interactive use with -it, but x11docker would not need that:

Yep, but the problem with that is the input streams are completely busted. Try doing it and then running docker-compose.exe up in a project, and then hit CTRL+C. Or, try running docker.exe with the -it flags.

Maybe this needs additional translation magic for line endings.

If this works, x11docker could automatically look for docker.exe if it runs in WSL.

---

Another comment explains how to set up docker in WSL using already existing docker-for-win TLS connection (without publishing an insecure TCP port). Looks a bit complicated at the first glance, but at least someone cared about a secure setup.

---

Just for doku, failing in my tests: How to set up docker natively in WSL without docker-for-win

[mviereck]

you would only need to create a bash alias

alias is unusable within scripts. x11docker now adds /c/Program Files/docker to PATH and runs docker.exe instead of docker in all of MSYS2/Cygwin/WSL. Hopefully this does not break anything.

---

WSL does not terminate Windows processes on Windows that were started within WSL although they disappear in WSL ps output after sending TERM.

I've included a workaround to get and kill the Windows pid. Though, it may be not reliable.

[mviereck]

Previous versions of VcXsrv had a bug in xauth.exe that forced x11docker to disable X authentication protocols with -ac. Since VcXsrv version 1.20.0.1 this xauth bug is fixed and honoured by x11docker. A warning is shown for previous versions of VcXsrv with a recommendation to update it.

This is one of the advantages of VcXsrv compared to Xming: It provides an xauth.exe and is actively maintained. After I've reported the bug, it was fixed soon.

[mviereck]

To adress the issue that drive C: is not allowed per default to be shared with docker containers, I've included a new option --homebasedir additionally to already existing --cachebasedir. --cachebasedir would always be needed to point to e.g. somewhere in drive D:. --homebasedir allows to specify a base directory to store persistant HOME folders for option --home. Already given option --homedir only allowes to specify HOME directory for one image, not a base directory for all.

[mviereck]

x11docker is now supposed to work flawlessly on MS Windows using MSYS2, Cygwin or WSL.

Most x11docker options should already work well. Exceptions:

• maybe in an upcoming release: --pulseaudio
  • will fail, maybe possible with some development: --sharessh
  • supposed to work, but may need a test: --stdin.
• impossible: --systemd, --alsa

Needs a test:

• --gpu: supposed to work in seamless mode (not in desktop mode) due to some instructions in the net for --xwin and --vcxsrv. It sets X options -wgl +iglx and environment variable LIBGL_ALWAYS_INDIRECT=1.

Please test --gpu with:

x11docker --gpu --verbose --no-entrypoint -- x11docker/xfce sh -c "'glxinfo | grep OpenGL'"

Output should contain the graphics card vendor, e.g. AMD, NVIDIA or Intel.

@1138-4EB : Could you please run a test in WSL? I would like to publish a new x11docker release 5.0.0 soon.

[mviereck]

Option --pulseaudio is now supported, too. It needs Cygwin installed in C:/cygwin64 and pulseaudio installed in Cygwin. It works for all of MSYS2 / Cygwin / WSL. It needs to be tested within container, though. I can only test for host applications in Win10 VM.

Pulseaudio daemon runs in Cygwin and allows TCP connections without authentication. That may be abused by others in the network to access audio input or output. Maybe I'll find a more secure solution some day. A warning message is shown. Multiple containers share the same connection. Pulseaudio daemon continues to run after x11docker terminates.

Edit: pulseaudio daemon can be killed with taskkill.exe /PID pulseaudio.exe /F

[eine]

You would only need to create a bash alias that translates command docker to /mnt/c/xyz/docker.exe.

It works. The full path is /mnt/c/Program\ Files/Docker/Docker/resources/bin/docker.exe.

-it fails because winpty is missing in this environment:

$ /mnt/c/Program\ Files/Docker/Docker/resources/bin/docker.exe run -it --rm alpine ls
the input device is not a TTY.  If you are using mintty, try prefixing the command with 'winpty'

NOTE: it is a pain to test WSL because copy and paste are not available :S.

---

Cygwin works ok with either --xwin or --vcxsrv.

MSYS is failing now. Vcxsrv is started and the container too, but nothing is shown. Exiting with Ctrl+C closes both the X server and the container properly. See x11docker_msys.log.

I just updated vcxsrv. Same behaviour. I will do some further test in about an hour.

[mviereck]

Thanks for testing!

Your VcXsrv version is 1.20.0.1 and is up to date already. Cookie creation should work, but has failed in your log. Cookie creation here works. Maybe I have to include a longer delay between start of VcXsrv and cookie creation. As a quick hack I will include a longer sleep, but think of a better solution to check whether VcXsrv is ready.

Edit: Looking closer at the log I found that a cookie creation failure check failed. x11docker tries to create a cookie in two steps. First it tries to retrieve one from X server, and if that fails, it creates one without the help of X. Retrieving from VcXsrv always fails here, but the check for this failure works. I've made an update that skips the first try to get a cookie from X server for --vcxsrv.

The full path is /mnt/c/Program\ Files/Docker/Docker/resources/bin/docker.exe.

This is surprising. Here it is /mnt/c/Program\ Files/Docker/docker.exe.

NOTE: it is a pain to test WSL because copy and paste are not available :S.

ok. ;-) Just one test if x11docker works at all, and I am happy. It runs docker.exe instead of docker to avoid wrong startup of possible /usr/bin/docker. Copy works selecting with left mouse button. Paste with right mouse button. Neither intuitive nor practical, but at least possible.

[eine]

MSYS with vcxsrv work now. I tested Cygwin again: both xwin and vcxsrv work. X servers and container are properly closed.

With WSL (defaults to --vcsxrv) it happens the same as with MSYS before: everything works ok, but nothing is shown. When I close it with Ctrl+C it takes longer that with wither MSYS or Cygwin to close everything (but it is properly closed). Therefore, I think that it might take even longer for the connection between vcxsrv and the 'VM' to be ready. See x11docker_wsl.log.

To adress the issue that drive C: is not allowed per default to be shared with docker containers, I've included a new option --homebasedir additionally to already existing --cachebasedir.

I tried ./x11docker --homebasedir /d/x11docker-home simexp/octave octave in MSYS:

• The full note (x11docker note: Per default x11docker stores its cache files on drive C:....) is shown, even though I set homebasedir.
• The path is not found, although it exists: x11docker WARNING: Option --homebasedir: Specified path does not exist: /d/x11docker-home.
  • If I use --homebasedir //d/x11docker-home it fails because it is translated to /c/msys64///d/x11docker-home.
  • /c/msys64/d/x11docker-home fails too.
  • D:/x11docker-home is the only one that seems to work. But then, from octave I cannot see any content. x11docker_homebasedir.log

Please test --gpu Output should contain the graphics card vendor, e.g. AMD, NVIDIA or Intel.

Is this a valid output?

OpenGL vendor string: Intel
OpenGL renderer string: Intel(R) HD Graphics 4400
OpenGL version string: 1.4 (4.3.0 - Build 20.19.15.4549)
OpenGL extensions:

x11docker_gpu.log

This computer has also an NVIDIA card, but I have never tried using it with docker.

Option --pulseaudio is now supported, too. It needs to be tested within container, though. I can only test for host applications in Win10 VM.

Any container you know to be ready to be used for testing?

Pulseaudio daemon runs in Cygwin and allows TCP connections without authentication. That may be abused by others in the network to access audio input or output. (...) Multiple containers share the same connection. Pulseaudio daemon continues to run after x11docker terminates.

Does this imply that others in the network can access audio input/output in the host even after containers are closed? Or it applies to containers only?

As a quick hack I will include a longer sleep, but think of a better solution to check whether VcXsrv is ready.

Using a container with xdpyinfo (progressively increasing the waiting time) until a valid output is obtained?

This is surprising. Here it is /mnt/c/Program\ Files/Docker/docker.exe.

This is version Version 18.06.0-ce-win72 (19098).

ok. ;-) Just one test if x11docker works at all, and I am happy. It runs docker.exe instead of docker to avoid wrong startup of possible /usr/bin/docker.

It's not a problem. But it takes longer XD.

Copy works selecting with left mouse button. Paste with right mouse button. Neither intuitive nor practical, but at least possible.

None of the buttons work. It seems that the mouse is not detected at all, so I cannot select. To paste, I tried both Ctrl+V and Shift+Insert, but none of them work.

[mviereck]

With WSL (defaults to --vcsxrv) it happens the same as with MSYS before: everything works ok, but nothing is shown.

There is a difference. The previous failure in MSYS was due to missing X cookie. octave falls back to CLI:

Authorization required, but no authorization protocol specified
octave: unable to open X11 DISPLAY
octave: disabling GUI features
Authorization required, but no authorization protocol specified

The log x11docker_wsl.log does not show an issue. The cookie is created, octave does not show an error. Maybe octave window was just hidden behind other windows instead of appearing on top? You can check with --desktop to be sure not to miss the X window.

Is this a valid output? OpenGL vendor string: Intel OpenGL renderer string: Intel(R) HD Graphics 4400

Yipiieh! :tada: Yes, it is valid. GPU acceleration is handled different than on Linux. On Linux x11docker shares devices in /dev/dri, on Windows all GPU requests are redirected to X server that has to handle this. On Linux direct rendering is standard (direct access to GPU), but VcXsrv and Xwin work with indirect rendering. Your test run uses --vcxsrv, you could test with --xwin, too. They are quite similar, I assume it will work, too.

This computer has also an NVIDIA card, but I have never tried using it with docker.

I don't know how this is handled on Windows. If you can somehow switch entire Windows to use NVIDIA, it should already work. If there is a wrapper application like bumblebee on Linux to run single applications with NVIDIA, it should work to run x11docker with that. Because the X server is handling GPU access, it should work without special setup for docker.

Using a container with xdpyinfo (progressively increasing the waiting time) until a valid output is obtained?

This is not an option as docker may need a password depending on setup, and also I don't want to load an additional image behind the scenes. I have included a check with xhost.exe that is delivered with VcXsrv. It seems to work here. I am not sure if that check is reliable and stable enough. The check waits up to 5 seconds, but it succeeds immediatly here.

This is version Version 18.06.0-ce-win72 (19098).

Here it is Docker version master-dockerproject-2018-06-14, build 805b3412. I'll add /mnt/c/Program\ Files/Docker/Docker/resources/bin/ to PATH to make sure it is accessable.

Any [pulseaudio] container you know to be ready to be used for testing?

A ready one would be x11docker/lxde-wine, but it is huge. (Though, you can run wine on Windows :smile: ). Rather test with:

FROM debian:stretch
RUN apt-get update && apt-get install -y --no-install-recommends pavucontrol
CMD pavucontrol

Does this imply that others in the network can access audio input/output in the host even after containers are closed? Or it applies to containers only?

It applies to all applications, all containers and every spy in your network. It uses the same IP as VcXsrv (10.0.75.1 in your case, 10.0.2.15 here). I am not sure about this IP; is it restricted to localhost access already? Otherwise the whole world may try to listen you, or it provides you some advertising through the speakers.

Edit: pulseaudio daemon can be killed with taskkill.exe /PID pulseaudio.exe /F

I tried ./x11docker --homebasedir

I'll look at that

---

One issue I have to fight with is to run Cygwin applications in MSYS2. Cygwin applications run well in cmd.exe and in WSL. But in MSYS2 it interferes with different cygwin.dll versions. I can run using cmd.exe /C cygwincommand in MSYS2, but than get issues with different mount points for C:. ( /c/ vs. /cygdrive/c/). I can change the mount point with mount --change-cygdrive-prefix /cygdrive, but than I get issues with PATH, and the mount point is changed for all parallel MSYS2 tasks, too. Is there a way to run cmd.exe in MSYS2 without influencing the environment of cmd.exe? Or is there a general way to run Cygwin applications in MSYS2? I did not find any hint in the net. A solution for this would help to set up authenticated --pulseaudio, too.

[eine]

Maybe octave window was just hidden behind other windows instead of appearing on top? You can check with --desktop to be sure not to miss the X window.

It works, although it takes quite long for octave to open after the desktop is shown. I then tried without --desktop and it works ok.

Your test run uses --vcxsrv, you could test with --xwin, too. They are quite similar, I assume it will work, too.

I tried both in Cygwin. Both work ok.

This is not an option as docker may need a password depending on setup, and also I don't want to load an additional image behind the scenes.

What about using xdyinfo locally instead of inside a container? At least in Cygwin and WSL is is available.

I'll add /mnt/c/Program\ Files/Docker/Docker/resources/bin/ to PATH to make sure it is accessable.

It works. No alias is required now.

pulseaudio daemon can be killed with taskkill.exe /PID pulseaudio.exe /F

Can this be added as warning note?

Is there a way to run cmd.exe in MSYS2 without influencing the environment of cmd.exe? Or is there a general way to run Cygwin applications in MSYS2? I did not find any hint in the net.

I never tried it. I normally use MSYS and I don't touch Cygwin at all.

---

Will update about pavucontrol later.

[mviereck]

I tried ./x11docker --homebasedir /d/x11docker-home simexp/octave octave in MSYS: The full note (x11docker note: Per default x11docker stores its cache files on drive C:....) is shown, even though I set homebasedir.

Ok, will be suppressed.

The path is not found, although it exists: x11docker WARNING: Option --homebasedir: Specified path does not exist: /d/x11docker-home.

I found a bug that missed to add /cygdrive or /mnt before the check in Cygwin or WSL. But that would not have been an issue in MSYS2. Maybe you accidently did the tests in Cygwin? Here all syntax variants work well.

D:/x11docker-home is the only one that seems to work. But then, from octave I cannot see any content. x11docker_homebasedir.log

This log shows you are using MSYS2. To have an effect for octave you need to add --home, too. Than a folder /d/x11docker-home/simexp-octave will be created and shared with container as its home directory. --homebasedir only specifies the parent directory where to store all home folders for different images. It has only visible effect along with --home. (To directly specify the home directory itself you can use --homedir DIR instead of --homebasedir DIR --home).

---

It works, although it takes quite long for octave to open after the desktop is shown.

About 5 seconds? That is the duration for X accessability check; maybe it failed somehow. You would see that in verbose output, 25 reiterations of x11docker: N. access check for --vcxsrv.

What about using xdyinfo locally instead of inside a container? At least in Cygwin and WSL is is available.

It is not available in MSYS2. x11docker in MSYS2 already has dependency VcXsrv and it makes sense to use the already given tools of VcXsrv. VcXsrv also provides xwininfo.exe. x11docker uses it now instead of xhost.exe for accessability check. It integrates nicer into the code (without a cmd.exe workaround that was needed for xhost.exe).

---

pulseaudio daemon can be killed with taskkill.exe /PID pulseaudio.exe /F Can this be added as warning note?

Already included.

[eine]

I tried x11docker --pulseaudio pavucontrol in MSYS, Cygwin and WSL (where pavucontrol is the image built with the Dockerfile you suggested above). Same behaviour:

The X server is properly started and the container too. A window with title Volume Control@<hash> is shown. It says 'Connection to PulseAudio failed. Automatic retry in 5s'. When I close the window a smaller error window is shown: 'Error writing config file %s/fakehome/f555l/.config/pavucontrol.ini: Invalid argument'. Both the container and the X server are properly closed.

x11docker_msys_audio.log x11docker_cygwin_audio.log

Already included.

It is not shown with the command above.

---

Ok, will be suppressed.

Did you modify it already? It is still shown.

Maybe you accidently did the tests in Cygwin?

It was in MSYS.

To have an effect for octave you need to add --home, too.

My bad. Now ./x11docker --homebasedir /d/x11docker-home/ --home simexp/octave octave works in MSYS! /cygdrive/d/x11docker-home/ works in Cygwin and /mnt/d/x11docker-home/ in WSL. Everything seems ok.

About 5 seconds? That is the duration for X accessability check; maybe it failed somehow.

It seems that the first time I try after starting the computer, it needs some more time and sometimes fails. But just trying again, it is quite fast.

[mviereck]

Argh, an option parsing error. Please try:

x11docker --pulseaudio=tcp pavucontrol

Edit: ok, is fixed now. Option --pulseaudio was not applied at all. Now it works again without =tcp. The possibility to add an optional argument to --pulseaudio was introduced in #71 to have a choice between TCP connection (--pulseaudio=tcp) or unix socket connection (--pulseaudio=socket). Without optional argument --pulseaudio defaults to socket on Linux and to tcp on Windows. On Windows only tcp is possible, on Linux both modes are supported.

---

Did you modify it already? It is still shown.

Now it is modified.

Now ./x11docker --homebasedir /d/x11docker-home/ --home simexp/octave octave works in MSYS! /cygdrive/d/x11docker-home/ works in Cygwin and /mnt/d/x11docker-home/ in WSL. Everything seems ok.

Great! btw, you would not need to add /cygwin or /mnt, x11docker adds that itself if missing. --homebasedir /d/x11docker-home should work in all of MSYS2/Cygwin/WSL. You can write --homebasedir D:/x11docker-home as well.

It seems that the first time I try after starting the computer, it needs some more time and sometimes fails. But just trying again, it is quite fast.

It is probably due to file caching. On first start the system has to load a lot of data, e.g. X server, docker image, and several helper applications. Once they are in cache (RAM or hard disk cache) , the system accesses them much faster.

[eine]

x11docker --pulseaudio pavucontrol in MSYS now shows 'Establishing connection to PulseAudio. Please wait...'. The same error as before when the window is closed. x11docker --pulseaudio=tcp pavucontrol produces the same result in cygwin. See x11docker_msys_pavuctrl.log.

[mviereck]

Everything looks ok in the log. The daemon is starting and environment variable PULSE_SERVER is set correctly.

Can you please run pavucontrol from host within Cygwin?

x11docker --pulseaudio --exe pavucontrol

And check for running daemon with:

tasklist.exe | grep pulseaudio

The same error as before when the window is closed.

This message seems to be harmless.

[eine]

pavucontrol was not installed in Cygwin. I had only installed pulseaudio. I added it now, and x11docker --pulseaudio --exe pavucontrol starts a daemon.

Still getting the same behaviour with x11docker --pulseaudio pavucontrol in Cygwin.

[mviereck]

I added it now, and x11docker --pulseaudio --exe pavucontrol starts a daemon.

Does pavucontrol from Cygwin establish a connection, i.e. shows several devices tabs and a volume control slider?

[eine]

At first nothing was shown at all. It complained because it could not connect to the X server in 30 seconds. Then I tried again and I can see tabs and sliders now. However, it is still not working if I remove the option --exe.

[mviereck]

However, it is still not working if I remove the option --exe.

Maybe there is some sort of automatic authentication that refused connections from IP's outside of localhost.

I've managed to set up authenticated TCP connection for --pulseaudio in MSYS2/Cygwin/WSL. The pulseaudio TCP module checks for container IP. Pulseaudio daemon still keeps running after terminating x11docker, but no unauthenticated modules are loaded anymore.

Please test if it works now:

x11docker --pulseaudio pavucontrol

If it fails, please try:

x11docker --pulseaudio --hostnet pavucontrol

If that fails, too, please try in Cygwin and/or WSL:

x11docker --pulseaudio --exe pavucontrol

---

It complained because it could not connect to the X server in 30 seconds.

This occured a few times here, too. Cygwin seems to crash. Unfortunately I've missed to get the logfile. If that happens more often, I'll have to investigate that.