no X11() in rstudio_stable ?

[kforner]

Hi,

I got here from the bioconductor docker, which is based upon rocker/rstudio, which seems to point here.

I realized that X11() was not enabled in the installed R, which makes sense in the context of rocker/rstudio, but less for a generic bioC docker. rstudio_stable is based upon rocker/r-ver:xxxx, which also seems to lack X11 capability.

Questions:

• is it expected / on purpose ?
• is there any a posteriori way to add X11 capability ?

Thanks.

[kforner]

from https://github.com/rocker-org/rocker-versioned/blob/master/r-ver/Dockerfile#L84 I can answer my own question: it is on purpose. It is a pity, docker X11 apps work perfectly on a X11-enabled host. I suppose it is to make it as lightweight as possible, which on one hand makes sense for such a "base" image, but on the other hand, with capabilities removed from the start, it restricts the kind of apps you can build upon.

[cboettig]

Hey Karl,

Yeah, like you guess I was just going for keeping the weight down; few of the use cases I'm familiar with had a compelling need for X11 even though it's possible (https://github.com/rocker-org/rocker/issues/110).

Still, maybe you're right that I should at least see how much it really saves to leave it out. Perhaps I can compile with X11 support and still remove those build-dep libraries after the fact without gaining much weight? Then at least a user could add them back on downstream. (Not 100% sure that I won't break anything if I compile X11 support in and remove those libs though?)

[eddelbuettel]

You maybe confusing R as it stores its compile-time configuration:

R> capabilities()
       jpeg         png        tiff       tcltk         X11        aqua 
       TRUE        TRUE        TRUE        TRUE        TRUE       FALSE 
   http/ftp     sockets      libxml        fifo      cledit       iconv 
       TRUE        TRUE        TRUE        TRUE       FALSE        TRUE 
        NLS     profmem       cairo         ICU long.double     libcurl 
       TRUE        TRUE        TRUE       FALSE        TRUE        TRUE 
R>  

Note the X11 in there.

[kforner]

Yeah, like you guess I was just going for keeping the weight down; few of the use cases I'm familiar > with had a compelling need for X11 even though it's possible (rocker-org/rocker#110).

plots ? GUIs ?

Still, maybe you're right that I should at least see how much it really saves to leave it out.

I think it could be worth the evaluation effort, because otherwise all the downstream uses are potentially limited, which is unfortunate for rocker/r which IMHO is supposed to be a reference R.

Perhaps I can compile with X11 support and still remove those build-dep libraries after the fact without gaining much weight? Then at least a user could add them back on downstream. (Not 100% sure that I won't break anything if I compile X11 support in and remove those libs though?)

Maybe just removing the dev packages (i.e. libXXX-dev) which contain the headers, which are no longer needed after compilation.

Another approach could be to make two different rocker R, one full-fledged, and one lightweight for specific uses ?

[eddelbuettel]

@kforner

Perhaps I can compile with X11 support and still remove those build-dep libraries after the fact without gaining much weight?

[eddelbuettel]

Anybody, I have no dog in this fight. 'versioned' is Carl's thing and I'll get out of the way. Happy Holidays.

[eddelbuettel]

Maybe X11 is always off under Docker unless you do extra work. It is your claim that this worked.

I am with Carl. I personally never use X11 with Docker. I am just demonstrating how to ask R for whether it thinks it has X11 -- and was raising my eyebrows when @cboettig suggested to compile-with only to remove-afterwards which strikes me as ... weird. Anyway, I am unsubscribing from this thread as I have nothing of value to add.

[kforner]

@eddelbuettel ok, happy holidays then !

[kforner]

ok. I found the proper way to reproduce it on my computer. The problem is that even if R has the X11 capability, but the setting is not right, or the access to the X11 server is not authorized, R asnwers FALSE.

So it actually works with rocker/r-devel:

xhost + # beware, potentially insecure
docker run --rm -ti -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix:0 rocker/r-devel R -q -e "capabilities()"
> capabilities()
       jpeg         png        tiff       tcltk         X11        aqua 
       TRUE        TRUE        TRUE        TRUE        TRUE       FALSE 
   http/ftp     sockets      libxml        fifo      cledit       iconv 
       TRUE        TRUE        TRUE        TRUE       FALSE        TRUE 
        NLS     profmem       cairo         ICU long.double     libcurl 
       TRUE        TRUE        TRUE        TRUE        TRUE        TRUE

but does not using r-ver (as expected):

xhost + # beware, potentially insecure
docker run --rm -ti -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix:0 rocker/r-ver:3.3.2 R -q -e "capabilities()"
> capabilities()
       jpeg         png        tiff       tcltk         X11        aqua 
       TRUE        TRUE        TRUE       FALSE       FALSE       FALSE 
   http/ftp     sockets      libxml        fifo      cledit       iconv 
       TRUE        TRUE        TRUE        TRUE       FALSE        TRUE 
        NLS     profmem       cairo         ICU long.double     libcurl 
      FALSE        TRUE        TRUE        TRUE        TRUE        TRUE 

I discovered the problem because it used to work for us previously, with a docker based on a bioc docker 3.2. Recently I upgraded to bioc 3.4, but the docker image had problems with apt-get. I rebuilt using the bioc Dockerfiles, but used rocker/rstudio:3.3.2 as the base image.

I really would like to be able to use a community-based docker, so that when we encounter a problem, it is easy and fast for people to reproduce it. I'd be glad to help you in any way if you need to.

[cboettig]

@kforner Right, I believe I can remove the -dev headers for the relevant X libraries and keep the runtime versions and things will be okay, as you've probably noticed that's what I've already done with a bunch of the graphics and compression libraries. (e.g. comparing the depends list of r-base-core vs its build-deps in debian). I'll take a look at that and see how much those run-time libraries would ask (or feel free to run that experiment!).

Re why I've excluded X11: for GUI/Graphics, I generally find using RStudio-server is the more popular way to do this. Docker is most often used on remote machines, and X11-forwarding often experiences significant latency that we avoid with RStudio-server. Moreover, rstudio-server approach is more platform independent -- linking the host X system can get all kinds of complicated, particularly for users not on a Linux-based host, (though maybe you've had more luck with people doing this than I have). Here's the alias I use to link the local X:

alias docker-X='XSOCK=/tmp/.X11-unix && XAUTH=/tmp/.docker.xauth && xauth nlist :0 | sed -e "s/^..../ffff/" | xauth -f $XAUTH nmerge - && docker run  -v $XSOCK:$XSOCK -v $XAUTH:$XAUTH -e XAUTHORITY=$XAUTH  -e DISPLAY=$DISPLAY --rm -it -P --user 1000 -v $(pwd):/home/rstudio/`basename $PWD` -w /home/rstudio/`basename $PWD`'

and then I run something like docker-X r-base but linking XAUTH, XSOCK and DISPLAY seems both cumbersome and fragile.

I do realize there are some packages that explicitly use X11-based interfaces and can't be run directly RStudio, I just don't use them. Are such packages the reason you want to link X11? or you just link it for graphics output? I also agree with your general point that it would be good to have a pretty general-purpose base image; as the rocker-issues show, others have an interest in X11.

Anyway, let's see what it adds to include the X11 runtime libs and compile with X11 support

[cboettig]

@kforner took a quick stab at this: https://github.com/rocker-org/rocker-versioned/blob/master/X11/Dockerfile, but for some reason building that Dockerfile, capabilities() still says X11 is FALSE, although R compiles fine without the --without-x11 flag and I am pretty sure I've included all the X11 libraries listed as r-base-core dependencies, and all the build-deps listed as build deps (compare to: https://github.com/rocker-org/rocker-versioned/blob/master/r-dependencies.md). Anyway feel free to play around with those scripts and see what you come up with.

[cboettig]

@kforner Okay, turns out X11 was functioning fine but the capabilities test I was using was failing (back to our discussion of why X11 is tricky, eh?).

Anyway, what I've done is to compile R with X11 support, so that a user can add the required runtime libraries downstream and have X11 functioning, but I don't ship them in r-ver image, which avoids any meaningful increase in the image size.

My X11 Dockerfile provides an example of just apt-getting the necessary runtime X11 dependencies on top of r-ver, showing what you would do downstream to add these in. Not sure if I'll make that Dockerfile an official part of the suite (currently the stack here is linear), easy enough to apt-get those seven package libs when needed.

Give it a whirl (once the images rebuild) and feel free to re-open if things don't look right to you.

[kforner]

Okay, turns out X11 was functioning fine but the capabilities test I was using was failing (back to our discussion of why X11 is tricky, eh?).

right, glad to see I'm not the only one who gets bitten ;)

Anyway, what I've done is to compile R with X11 support, so that a user can add the required runtime libraries downstream and have X11 functioning, but I don't ship them in r-ver image, which avoids any meaningful increase in the image size.

That seems a perfect solution !

My X11 Dockerfile provides an example of just apt-getting the necessary runtime X11 dependencies on top of r-ver, showing what you would do downstream to add these in.

ok

Not sure if I'll make that Dockerfile an official part of the suite (currently the stack here is linear), easy enough to apt-get those seven package libs when needed. Give it a whirl (once the images rebuild) and feel free to re-open if things don't look right to you.

Unfortunately I won't be able to test/use it before next year, but then I'll use it on a daily basis. Thank you very much and happy new year !!

[kforner]

It works perfectly, thanks a lot @cboettig !! :clap:

[DillonHammill]

@cboettig, does this approach work with rocker/r-verse:devel? My package requires X11() calls to function correctly.

This is the current setup in my Dockerfile:

FROM ubuntu:16.04

# liprotobuf version 2.6.0
RUN apt-get update
RUN apt-get install libprotobuf-dev -y

# install dependencies
RUN apt-get update && apt-get install -y\
    autoconf \
    automake \
    libtool \
    libxml2 \
    libhdf5-dev

# R 
FROM rocker/verse:devel

# Install X11 dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    libx11-6 \
    libxss1 \
    libxt6 \
    libxext6 \
    libsm6 \
    libice6 \
    xdg-utils \
  && rm -rf /var/lib/apt/lists/*

# install packages
RUN R -e "BiocManager::install(version = 'devel')" 
RUN R -e "BiocManager::install('flowCore')" 
RUN R -e "BiocManager::install('flowWorkspace')" 
RUN R -e "BiocManager::install('openCyto')" 
RUN R -e "devtools::install_github('DillonHammill/CytoExploreRData')" 
RUN R -e "devtools::install_github('DillonHammill/CytoExploreR')"

I have included the X11 dependencies as mentioned above.

I have then built the image in a docker container using:

docker run --rm -dit ^
    -p 8787:8787 ^
    -v /tmp/.X11-unix:/tmp/.X11-unix ^
    -e DISPLAY=$DISPLAY ^
    -e PASSWORD=cytoexplorer ^
    --name=cytoexplorer-rstudio ^
    dhammill/cytoexplorer

In the browser, I can go to localhost:8787 and login to RStudio. If I run capabilities() in that RStudio session X11 is FALSE.

Is there something that I am missing? Do I need to get R and RStudio separately?

Any advice would be greatly appreciated.

Dillon

[cboettig]

@DillonHammill just following up here to loop back to our email thread. As you've seen, the details will depend on your host OS, but I think you're missing some bindings even for linux, see our X11/README

docker build -t rocker/x11 .
XSOCK=/tmp/.X11-unix && XAUTH=/tmp/.docker.xauth && xauth nlist :0 | sed -e "s/^..../ffff/" | xauth -f $XAUTH nmerge - && docker run  -v $XSOCK:$XSOCK -v $XAUTH:$XAUTH -e XAUTHORITY=$XAUTH  -e DISPLAY=$DISPLAY --rm -it rocker/x11 R -e "capabilities()" 

I believe most of those sockets are Linux specific, you may need different paths on Mac and I have no idea what it would be on Windows, but you should be able to look for generic advice on X11+docker as most of this isn't Rocker-specific.

In general we try and avoid the need for X11 bindings whenever possible because it requires a lot of host-specific configuration external to the container, which isn't really in the docker spirit. Fortunately more packages that require GUI interfaces are adopting alternatives like RStudio extensions interface which is more compatible with a containerized approach. HTH