X11 xauth

[morrisondan]

I couldn't get my X11 subuser apps to work initially. Then I used xhost +. Then I found the right way to give the subuser container permission to the host's display:

1. on Linux host, COOKIE=$(xauth list $DISPLAY | awk '{ print $NF }')
2. in subuser container, export XAUTHORITY=$HOME/.Xauth_subuser touch "$XAUTHORITY" xauth add $DISPLAY . $COOKIE

Now: was X11 access suppose to work out of the box? If not, what's a good way to pass $COOKIE from host to subuser and make sure the subuser container runs the above setup before running its main executable?

[timthelion]

X11 works out the box on my debian wheezy system, as well as on the ubuntu systems of several people who have tried subuser. This is the first I hear of needing to play with xauth and xhost. I will try to look into it some, but really, x11 is not something I want to use long term as the main means for displaying windows with subuser. It is simply insecure.

The goal is to use XPRA. However, this has been slowed by the fact that I would like to do so without having to acutally install xpra into the container or onto the host. Rather, I would like to run it in paralel containers and continue to use the current system to connect those paralel containers up.

[timthelion]

I will admit, however, that I have run into issues connecting to X11 when X11 was launched using say x11vnc or ssh. Can you post error messages? Can you also share with me whether you are using startx or some display manager like gdm, lightdm ect?

[timthelion]

While your method is straight forward enough to implement in subuser, reconfiguring the systems xhost settings to make them more permissive seems like an undesirable thing for a security product to do ;).

[morrisondan]

The subuser container is launched from a screen session over ssh, and tries to connect to :15 (which is an xpra display I keep open). The message for xterm is No protocol specified xterm: Xt error: Can't open display: unix:15 You are right, with :0 (even over ssh / screen) there are no errors (though I have no ideea what shows up on the remote server). I hadn't even thought of trying that :) I hope you do keep direct X connections as an option, any remote desktop technology has its inconvenience and performance issues.

[timthelion]

I will continue to maintain the direct X11 access. If you look at the standard "x11" remains a perission which can be granted, while a new "gui" should appear which is more secure.

[morrisondan]

My method doesn't involve xhost, it replaces the insecure xhost method, which I also dislike; if you give an X client the MIT magic cookie, it can connect securely to the corresponding display. That's the beauty of it :)

But my question is, how could I automate the passing of $COOKIE and the running of the xauth setup commands in the container (before the subuser executable configured in permissions.json)?

[timthelion]

Aha, I have just read your comment again. I will put together a patch shortly and test out your method ;)

[morrisondan]

Well, if this is not normally needed, and X works out of the box, then I'm not sure the patch should be active by default on all executions... How will you even test it if it works for you :) ?

[timthelion]

I will use ssh like you do.

Or you can write the patch ;) . It is very simple.

1. You need to conditionally build the xauth file ON THE HOST (because we don't know if the xauth command exists inside the container.) You would would do this during the execution of the run function https://github.com/subuser-security/subuser/blob/master/logic/subuserlib/classes/runtime.py#L113 You need to build this xauth file somewhere on the host machine, I sugest you use python's built in tempdir functions. You should only create this file when the x11 permission is set, if self.getSubuser().getPermissions()["x11"]:. You'll want to do your temp file creation in the __init__ funciton of the runtime class so that it's path will be accessible to all functions: https://github.com/subuser-security/subuser/blob/master/logic/subuserlib/classes/runtime.py#L26
2. You need to mount that authentification file and set the XAUTHORITY env var on this line https://github.com/subuser-security/subuser/blob/master/logic/subuserlib/classes/runtime.py#L86 .

[morrisondan]

The whole point of running xauth add in the container is that only the container knows the proper hostname... Before arriving at this solution, I have tried copying the .Xauthority file to the container as-is, or using xauth merge (which is very similar) -- but these didn't work, because the hostnames didn't match, so the X client just ignored those database entries.

Now, xauth is an xorg, xbase-clients, xinit and xvfb dependency (on Debian) so I don't think it would be missing on an X11-enabled container (but the script can abort if the command is not present).

[morrisondan]

By the way, can the user tell subuser run to pass some envvars (like docker -e)? I could then change the permission.js executable to run the setup first.

Maybe putting the setup commands in SubuserImagefile would work too? Using docker's ONBUILD?

[timthelion]

I don't think that this is something that should go in the SubuserImageFile for two reasons. One is, that subuser aims to make it simple to package programs for it. If at all possible, this should be automagic, and I am 100% certain that it is possible to make it automagic if it is possible to do so in the SubuserImageFile. Secondly, the contents of the XAUTHORITY file will change over time, whereas the built image does not.

At this time, there are no preparation scripts that run within the image at run time. The closest we have is the preparation of images to be run https://github.com/subuser-security/subuser/blob/master/logic/subuserlib/runReadyImages.py. However, this is still not run every time a subuser is started, but rather every time a subuser's permissions.json file is updated.

That said, there is no good reason why it should not be possible to run a script in the container before the main executable is run. Indeed subuser used to do just this, before the runReadyImages code was added. It will require some care, I think that the script is best written in POSIX compliant sh. It can be included in source form within the subuser source repository, mounted as a read only volume to somewhere inside the container and injected here: https://github.com/subuser-security/subuser/blob/master/logic/subuserlib/classes/runtime.py#L104

[prahal]

About xauth and hostname , http://stackoverflow.com/a/25280523 tell to set the xauth entry authentication mode to FamilyWild "so that hostname does not matter". This via stream edit.

    XAUTH=/tmp/.docker.xauth
    xauth nlist :0 | sed -e 's/^..../ffff/' | xauth -f $XAUTH nmerge -
    docker run -ti -v $XSOCK:$XSOCK -v $XAUTH:$XAUTH -e XAUTHORITY=$XAUTH xeyes

Note that DISPLAY is set in the Dockerfile there.

[timthelion]

@prahal ooo thanks! This looks good :) I guess the script is going to need to be fiddled with a bit, but it should be quite easy to implement this.

[timthelion]

Can someone tell me a linux distro in which the xauth method is needed so that I can test out the change?

[timthelion]

Sorry it took me so long, this should be fixed now.

[Happy-Ferret]

I seem to run into the very bug this issue was about.

After installing the iceweasel-java@default image and trying to run it via "subuser run firefox", I run into the following error message.

IOError: [Errno 2] No such file or directory: u'/home/superuser/.subuser/volumes/x11/!service-subuser-firefox-xpra-client/subuser/.Xauthority'

Unfortunately, following the steps outlined by morrisondan does not work for me. Neither does copying my local .Xauthority to the container directory pointed out in the error message.

[timthelion]

morrisondan's workaround is no longer needed. Are you running subuser from pypi or from git? The version in git is much better and it is possible that there was a bug in the "stable" version which has now been fixed. Can you copy the entire output of the command please? There should be a traceback and not just an error line.

[Happy-Ferret]

Here you go.

No matches found, authority file "/home/superuser/.subuser/volumes/x11/!service-subuser-firefox-xpra-client/subuser/.Xauthority" not written
Traceback (most recent call last):
  File "/usr/local/bin/subuser-run.py", line 48, in <module>
    run(sys.argv)
  File "/usr/local/bin/subuser-run.py", line 42, in run
    runtime.run(argsToPassToImage)
  File "/usr/local/lib/python2.7/dist-packages/subuserlib/classes/subuserSubmodules/run/runtime.py", line 263, in run
    return reallyRun()
  File "/usr/local/lib/python2.7/dist-packages/subuserlib/classes/subuserSubmodules/run/runtime.py", line 245, in reallyRun
    self.getSubuser().getX11Bridge().addClient()
  File "/usr/local/lib/python2.7/dist-packages/subuserlib/classes/service.py", line 80, in addClient
    serviceStatus = self.start(serviceStatus)
  File "/usr/local/lib/python2.7/dist-packages/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 172, in start
    serviceStatus["xpra-client-service-cid"] = clientRuntime.run(args=clientArgs).getId()
  File "/usr/local/lib/python2.7/dist-packages/subuserlib/classes/subuserSubmodules/run/runtime.py", line 263, in run
    return reallyRun()
  File "/usr/local/lib/python2.7/dist-packages/subuserlib/classes/subuserSubmodules/run/runtime.py", line 239, in reallyRun
    self.setupXauth()
  File "/usr/local/lib/python2.7/dist-packages/subuserlib/classes/subuserSubmodules/run/runtime.py", line 195, in setupXauth
    with open(self.getXautorityFilePath(),"rb") as xauthFile:
IOError: [Errno 2] No such file or directory: u'/home/superuser/.subuser/volumes/x11/!service-subuser-firefox-xpra-client/subuser/.Xauthority'

And yes. I'm running the pip version.

[timthelion]

What exact operating system version are you using? The problem is actually that xauth is failing to connect to the X11 server:

No matches found, authority file "/home/superuser/.subuser/volumes/x11/!service-subuser-firefox-xpra-client/subuser/.Xauthority" not written

That is the output of the command generated here: https://github.com/subuser-security/subuser/blob/master/logic/subuserlib/classes/subuserSubmodules/run/runtime.py#L229

[Happy-Ferret]

I'm running a fresh install of Ubuntu 15.04 x64 Server Edition hosted inside a Virtualbox VM.

The only packages I installed manually are xinit, gnome-music, fvwm, docker/docker.io and python-pip.

[timthelion]

OK, I'm downloading the image now.

[timthelion]

I have installed a vm as described, however, I was unable to reproduce your issue.

Can you please try running the command:

$ xauth extract output $DISPLAY

And sending me the contents of the output file which is created?

[Happy-Ferret]

I re-installed the whole system. There was something very wrong with it. xinit wouldn't even produce an .Xauthority file and xauth extract would fail to run.

I feel like I'm now one step closer to solving the problem. However, when I'm running subuser run xterm (or firefox) now, subuser will try to constantly connect to the bridge before the recursions, eventually, exceed the stack, crashing the app.

This is what the traceback looks like. I shortened it a bit, since the beginning is nothing but a giant recursion anyways.

  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 167, in clearAndTryAgain
    self.createAndSetupSpecialVolumes()
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 176, in createAndSetupSpecialVolumes
    mkdirs(self.getServerSideX11Path())
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 173, in mkdirs
    clearAndTryAgain()
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 167, in clearAndTryAgain
    self.createAndSetupSpecialVolumes()
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 176, in createAndSetupSpecialVolumes
    mkdirs(self.getServerSideX11Path())
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 173, in mkdirs
    clearAndTryAgain()
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 166, in clearAndTryAgain
    self.cleanUp()
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 162, in cleanUp
    self.getUser().getDockerDaemon().execute(["run","--rm","--volume",os.path.join(self.getUser().getConfig()["volumes-dir"],"xpra")+":/xpra-volume","--entrypoint","/bin/rm",self.getServerSubuser().getImageId(),"-rf",os.path.join("/xpra-volume/",self.getSubuser().getName())])
  File "/home/superuser/subuser/logic/subuserlib/classes/docker/dockerDaemon.py", line 235, in execute
    return subuserlib.docker.run(args,cwd=cwd)
  File "/home/superuser/subuser/logic/subuserlib/docker.py", line 54, in run
    return subprocessExtras.call([getAndVerifyExecutable()]+args,cwd)
  File "/home/superuser/subuser/logic/subuserlib/docker.py", line 32, in getAndVerifyExecutable
    executable = getExecutable()
  File "/home/superuser/subuser/logic/subuserlib/docker.py", line 22, in getExecutable
    if subuserlib.executablePath.which("docker.io"): # Docker is called docker.io on debian.
  File "/home/superuser/subuser/logic/subuserlib/executablePath.py", line 36, in which
    programMatches = queryPATH(matchesImage)
  File "/home/superuser/subuser/logic/subuserlib/executablePath.py", line 57, in queryPATH
    appendIfMatches(exeFile)
  File "/home/superuser/subuser/logic/subuserlib/executablePath.py", line 49, in appendIfMatches
    if isExecutable(exeFile):
  File "/home/superuser/subuser/logic/subuserlib/executablePath.py", line 18, in isExecutable
    return os.path.isfile(fpath) and os.access(fpath, os.X_OK)
  File "/usr/lib/python2.7/genericpath.py", line 40, in isfile
    return stat.S_ISREG(st.st_mode)
RuntimeError: maximum recursion depth exceeded

[timthelion]

How comfortable are you editing code? It would be helpful if you were to comment out lines 169 and 171-175 here: https://github.com/subuser-security/subuser/blob/master/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py#L169 so that I could see the exception that is getting raised that is preventing makedirs from succeeding.

[Happy-Ferret]

Here you go.

Traceback (most recent call last):
  File "/home/superuser/subuser/logic/subuserCommands/subuser-run.py", line 54, in <module>
    run(sys.argv)
  File "/home/superuser/subuser/logic/subuserCommands/subuser-run.py", line 48, in run
    runtime.run(argsToPassToImage)
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/runtime.py", line 290, in run
    return reallyRun()
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/runtime.py", line 269, in reallyRun
    self.getSubuser().getX11Bridge().addClient()
  File "/home/superuser/subuser/logic/subuserlib/classes/service.py", line 98, in addClient
    serviceStatus = self.start(serviceStatus)
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 193, in start
    self.createAndSetupSpecialVolumes()
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 176, in createAndSetupSpecialVolumes
    mkdirs(self.getServerSideX11Path())
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/x11Bridge.py", line 170, in mkdirs
    self.getUser().getEndUser().makedirs(directory)
  File "/home/superuser/subuser/logic/subuserlib/classes/endUser.py", line 70, in makedirs
    os.mkdir(pathBeingBuilt)
OSError: [Errno 13] Permission denied: '/home/superuser/.subuser/volumes/xpra/xterm'

[timthelion]

So, the problem is that the folder /home/superuser/.subuser/volumes/xpra/xterm cannot be created. Does it exist already? What is the output of:

echo $UID
echo $USER
ls -la /home/superuser/.subuser/volumes/xpra/xterm
ls -la /home/superuser/.subuser/volumes
la -la /home/superuser/.subuser

?

[Happy-Ferret]

UID:  1000
USER:  superuser

/home/superuser/.subuser/volumes/xpra/xterm

total 16
drwxr-xr-x 4 root root 4096 Dez 10 23:53 .
drwxr-xr-x 3 root root 4096 Dez 10 23:53 ..
drwxr-xr-x 3 root root 4096 Dez 10 23:53 tmp
drwxr-xr-x 2 root root 4096 Dez 10 23:53 xpra-home

/home/superuser/.subuser/volumes

total 12
drwxr-xr-x 3 root      root      4096 Dez 10 23:27 .
drwxrwxr-x 9 superuser superuser 4096 Dez 10 23:27 ..
drwxr-xr-x 3 root      root      4096 Dez 10 23:53 xpra

/home/superuser/.subuser

total 44
drwxrwxr-x  9 superuser superuser 4096 Dez 10 23:27 .
drwxr-xr-x 19 superuser superuser 4096 Dez 11 18:01 ..
drwxrwxr-x  2 superuser superuser 4096 Dez 10 23:27 bin
drwxrwxr-x  3 superuser superuser 4096 Dez 10 23:27 homes
-rw-rw-r--  1 superuser superuser 1228 Dez 10 23:27 installed-images.json
-rw-rw-r--  1 superuser superuser    2 Dez 10 23:27 locked-subusers.json
drwxrwxr-x  3 superuser superuser 4096 Dez 10 23:27 locks
drwxrwxr-x  4 superuser superuser 4096 Dez 10 23:27 registry
drwxrwxr-x  3 superuser superuser 4096 Dez 10 23:17 repositories
drwxrwxr-x  5 superuser superuser 4096 Dez 10 23:27 runtime-cache
drwxr-xr-x  3 root      root      4096 Dez 10 23:27 volumes

This is the most recent traceback, after temporarily chowning the whole directory structure to superuser.

  File "/home/superuser/subuser/logic/subuserCommands/subuser-run.py", line 54, in <module>
    run(sys.argv)
  File "/home/superuser/subuser/logic/subuserCommands/subuser-run.py", line 48, in run
    runtime.run(argsToPassToImage)
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/runtime.py", line 290, in run
    return reallyRun()
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/runtime.py", line 270, in reallyRun
    command = self.getCommand(args)
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/runtime.py", line 184, in getCommand
    flags.extend(flagGenerator(permissions[permission]))
  File "/home/superuser/subuser/logic/subuserlib/classes/subuserSubmodules/run/runtime.py", line 121, in <lambda>
    ("access-working-directory", lambda p: ["-v="+os.getcwd()+":/pwd:rw","--workdir=/pwd"] if p else ["--workdir="+self.getSubuser().getDockersideHome()]),
OSError: [Errno 2] No such file or directory

[Happy-Ferret]

Never mind.

I forgot uncommenting some lines. Works now. Albeit probably not with the permissions it should run with.

[timthelion]

Which permissions should it run with? I don't understand.

[Happy-Ferret]

Well. I chowned all of .subuser/volumes and its subdirectories to superuser. Is that how it's supposed to be or are there certain directories that shouldn't be accessible to the regular user?

Truth be told, I haven't read up on the subuser standard yet.

Anyhow. It works now.

[timthelion]

I'm glad that it works. The permissions should not have been root. Docker creates shared volumes if the folders don't exist already, and Docker runs as root. That's why it was set to root, not due to some intention on the part of subuser. The only thing that is intentionally root in volumes is the .X11-unix socket folder. That has to be owned by root.

[Happy-Ferret]

Good to know that the .X11-unix socket folder ought to be owned by root. Changing this back then.

Thanks for your great product!

PS: Unrelated to subuser but I'm not really having a blast with xpra lately.

There's some breakage with more sophisticated software. Mostly due to the fact that an xpra session doesn't know about the underlying display system's full resolution (it also doesn't know how to handle hidden windows, the sort of Xulrunner apps can produce. But that will probably require patching on the part of a Xulrunner app).

I'm currently conceptualising a way to properly and generically communicate information such as this to an application running inside an xpra session.

Hopefully there is a proper way to make this happen (and not just on application start but also resolution changes). I have high hopes for using xpra (and possibly subuser) as part of my little software stack.