Miscellaneous things that would be useful

wch commented 10 years ago

Glad you guys are working on this! Sorry to file this as a single issue, but I figured it would be less annoying than filing them as separate issues, at least for now.

Here are some things that would be useful -- for my use cases, at least:

Nightly builds of an r-devel image
Images for package development, checking, and building, which include latex (I suppose instaling tetex should happen early on in the docker file, before installing things like RStudio and R-devel, which change frequently)
It would also be handy to have a way to log into a running container, because sometimes you need to poke around. (Though maybe this isn't necessary, because it sounds like nsenter is the way to go in the future.)

eddelbuettel commented 10 years ago

Hi Winston In order:

we are not yet that concerned with productionalizing this, but a nightly cronjob could be put on top of this; right now each commit triggers builds of all (!!) container and we're up to 10... -- so may make sense to split this off
it's called texlive these days, rather than tetex, and we do install a bunch in r-devel and hadleyverse
you can use containers interactively by launching with -i -t and making the command /bin/bash

cboettig commented 10 years ago

Hi Winston,

Thanks for the feedback / use cases, very helpful.

re entering the container; right, sometimes you'll want to enter the container that is running your RStudio instance as a persistent daemon, rather than just launching an interactive container. I agree that nsenter is entirely the way to go here.

I put this in my .bashrc:

function dock { sudo nsenter -m -u -n -i -p -t `docker inspect --format {{.State.Pid}} "$1"` /bin/bash; }

And then I can just use dock rstudio to enter the container (as root), provided I launched it with the flag --name=rstudio.

Re order of installation -- Yeah, haven't thought too much about leveraging caching. Note that we have to specify the RStudio .deb explicitly, so it's not grabbing the latest version necessarily. Lemme know if there's a clever way around that. At the moment all the images build in a reasonable time and on Docker Hub's servers anyway. And caching builds can be subtle, e.g. how to let docker know which layers you want rebuilt.

wch commented 10 years ago

Thanks for the tip about using nsenter. Now if only nsenter were simpler to install on host machines...

For installation order, it's not just build time -- it's more about the time that it takes for users to download an updated image. For example, if you first install RStudio and then texlive in an image, then update RStudio and rebuild the image, users will have to download gigabytes more data than if you had installed texlive and then RStudio.

I imagine that doing things this way would result in even more Dockerfiles though. Pretty soon you'll need a Makefile to build all the Dockerfiles. :)

cboettig commented 10 years ago

@wch thanks for clarifying, that's a good point. We don't really optimize not changing layers within an image, e.g. we've opted to run apt-get upgrade at the top of each container. Like you say, that can get complicated quickly. Definitely worth thinking about.

Re. installing nsenter -- why not just run it as a docker container?

docker run -v /usr/local/bin:/target jpetazzo/nsenter

wch commented 10 years ago

@cboettig I did actually install nsenter with that method. It's just that it has to download tons of stuff (the image size is 353 MB) just to compile a ~1MB binary. When you're spinning up VM hosts, it would be nice to make that more efficient.

eddelbuettel commented 10 years ago

But if you want a more lightweight r-devel .... then nothing beats building it locally. Just sayin' ...

wch commented 10 years ago

I'm told that you can get the latest RStudio version by looking at this URL: https://s3.amazonaws.com/rstudio-server/current.ver

And with that info, you can get the URL for latest RStudio build.

eddelbuettel commented 10 years ago

Very nice as I just mentioned to Carl over in another thread -- thanks for that!

eddelbuettel commented 10 years ago

Regarding point one, have a look at this auto-build.

I'll run it manually for a few days, and may then crontab it.

wch commented 10 years ago

Excellent!

eddelbuettel commented 10 years ago

The 'trick' attempted to shrink he nighly r-devel build worked -- eddelbuettel/drd now comes in at under 800mb which is not too shabby.

wch commented 10 years ago

Nice! What was the trick?

eddelbuettel commented 10 years ago

Single RUN command so that the files "disappear" before the AUFS layer is created. Because once layered, space can currently not be reclaimed. More discussion in this thread I started on docker-user.

cboettig commented 10 years ago

@wch Seems like Docker has granted your wish for a better alternative to nsenter: http://blog.docker.com/2014/10/docker-1-3-signed-images-process-injection-security-options-mac-shared-directories/

With Docker 1.3 you can now just run docker exec -it <container> bash to enter the container.

rocker-org / rocker

Miscellaneous things that would be useful #10