search

bgruening / docker-galaxy-stable

:whale::bar_chart::books: Docker Images tracking the stable Galaxy releases.
http://bgruening.github.io/docker-galaxy-stable
MIT License
226 stars 133 forks source link

IE does not work #127

Closed mictadlo closed 8 years ago

mictadlo commented 8 years ago

Hello, I am still no able to use IE (Rstudio and Ipython). The logs show the following:

127.0.0.1 - - [30/Dec/2015:01:55:06 +0000] "GET /visualization/show/rstudio?dataset_id=f2db41e1fa331b3e HTTP/1.1" 200 - "http://localhost/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0" [pid: 189|app: 0|req: 10/15] 172.17.0.1 () {48 vars in 928 bytes} [Wed Dec 30 01:55:06 2015] GET /visualization/show/rstudio?dataset_id=f2db41e1fa331b3e => generated 1559 bytes in 1077 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 2) 127.0.0.1 - - [30/Dec/2015:01:55:07 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/prng4.js HTTP/1.1" 404 - "http://localhost/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0" 127.0.0.1 - - [30/Dec/2015:01:55:07 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/rng.js HTTP/1.1" 404 - "http://localhost/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0" [pid: 189|app: 0|req: 12/16] 172.17.0.1 () {52 vars in 1041 bytes} [Wed Dec 30 01:55:07 2015] GET /plugins/interactive_environments/rstudio/static/js/crypto//rng.js => generated 304 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 1) [pid: 189|app: 0|req: 12/17] 172.17.0.1 () {52 vars in 1045 bytes} [Wed Dec 30 01:55:07 2015] GET /plugins/interactive_environments/rstudio/static/js/crypto//prng4.js => generated 306 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 3) 127.0.0.1 - - [30/Dec/2015:01:55:07 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/rsa.js HTTP/1.1" 404 - "http://localhost/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0" [pid: 189|app: 0|req: 13/18] 172.17.0.1 () {52 vars in 1041 bytes} [Wed Dec 30 01:55:07 2015] GET /plugins/interactive_environments/rstudio/static/js/crypto//rsa.js => generated 304 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0) 127.0.0.1 - - [30/Dec/2015:01:55:07 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/jsbn.js HTTP/1.1" 404 - "http://localhost/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0" [pid: 188|app: 0|req: 6/19] 172.17.0.1 () {52 vars in 1044 bytes} [Wed Dec 30 01:55:07 2015] GET /plugins/interactive_environments/rstudio/static/js/crypto//jsbn.js => generated 305 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 1) 127.0.0.1 - - [30/Dec/2015:01:55:07 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/base64.js HTTP/1.1" 404 - "http://localhost/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0" [pid: 189|app: 0|req: 14/20] 172.17.0.1 () {52 vars in 1047 bytes} [Wed Dec 30 01:55:07 2015] GET /plugins/interactive_environments/rstudio/static/js/crypto//base64.js => generated 307 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 2) 127.0.0.1 - - [30/Dec/2015:01:55:07 +0000] "GET /plugins/interactive_environments/rstudio/static/js/rstudio.js HTTP/1.1" 404 - "http://localhost/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0" [pid: 189|app: 0|req: 15/21] 172.17.0.1 () {52 vars in 1035 bytes} [Wed Dec 30 01:55:07 2015] GET /plugins/interactive_environments/rstudio/static/js//rstudio.js => generated 301 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 1) galaxy.web.base.interactive_environments INFO 2015-12-30 01:56:03,145 Starting docker container for IE ipython with command [docker run --sig-proxy=true -e DEBUG=false -e "GALAXY_URL=http:/" -e "CORS_ORIGIN=http://localhost" -e "HISTORY_ID=f597429621d6eb2b" -e "NOTEBOOK_PASSWORD=none" -e "DATASET_HID=None" -e "PROXY_PREFIX=" -e "API_KEY=190fc21dcc3277504939d12b8767b8b8" -e "REMOTE_HOST=127.0.0.1" -e "USER_EMAIL=m.lorenc@qut.edu.au" -d -P bgruening/docker-ipython-notebook:15.10.1] galaxy.web.base.interactive_environments DEBUG 2015-12-30 01:56:03,648 Container id: fd11f6b411a2c81e5b343bee42d6ae26298d3b4507b23e5397bd1ecd2f6deb6b

galaxy.web.base.interactive_environments INFO 2015-12-30 01:56:03,648 Inspecting docker container fd11f6b411a2c81e5b343bee42d6ae26298d3b4507b23e5397bd1ecd2f6deb6b with command [docker inspect fd11f6b411a2c81e5b343bee42d6ae26298d3b4507b23e5397bd1ecd2f6deb6b ] 127.0.0.1 - - [30/Dec/2015:01:56:03 +0000] "GET /visualization/show/ipython?dataset_id=f2db41e1fa331b3e HTTP/1.1" 200 - "http://localhost/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0" [pid: 188|app: 0|req: 7/22] 172.17.0.1 () {48 vars in 928 bytes} [Wed Dec 30 01:56:03 2015] GET /visualization/show/ipython?dataset_id=f2db41e1fa331b3e => generated 1366 bytes in 661 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 3)

What did I do wrong?

Mic

bgruening commented 8 years ago

You are running Galaxy 15.10? Do you have your proxy started? Can you provide us with the proxy log? Your Galaxy URL looks wrong: "GALAXY_URL=http:/" you can change this in ipython.ini. Moreover you can try to access Galaxy with your real hostname instead of localhost.

mictadlo commented 8 years ago

Yes, I am running 15.10. Where can I find the Apache Proxy log on Ubuntu 14.04? How do I log in to the container and where is ipython.ini located?

bgruening commented 8 years ago

There is no Apache proxy. This container is using nginx. Can you please post the command you used to start the container?

mictadlo commented 8 years ago

Hi @bgruening, I am using Linux Mint 17.2 and docker 1.9.1 in VirtualBox. The host system is CentOS 6.6.

With DOCKER_PARENT=True

docker run -p 8080:80 -p 8021:21 -p 8800:8800 --privileged=true -e DOCKER_PARENT=True -v /var/run/docker.sock:/var/run/docker.sock -v /home/galaxy/galaxy_storage/:/export/ -e "GALAXY_CONFIG_ADMIN_USERS=m..." -e "host=0.0.0.0" bgruening/galaxy-stable
munged: Warning: Logfile is insecure: group-writable permissions set on "/var/log"
Enable Galaxy Interactive Environments.
...

Ipython

galaxy.web.base.interactive_environments INFO 2016-01-02 03:32:49,066 Starting docker container for IE ipython with command [docker run --sig-proxy=true -e DEBUG=false -e "GALAXY_URL=http:/" -e "CORS_ORIGIN=http://localhost:8080" -e "HISTORY_ID=f597429621d6eb2b" -e "NOTEBOOK_PASSWORD=none" -e "DATASET_HID=None" -e "PROXY_PREFIX=" -e "API_KEY=866760228e04dcb00e88ef3c0c82e6a7" -e "REMOTE_HOST=172.17.0.1" -e "USER_EMAIL=m.lorenc@qut.edu.au" -d -P   bgruening/docker-ipython-notebook:15.10.1]
galaxy.web.base.interactive_environments DEBUG 2016-01-02 03:32:49,542 Container id: 81bdfb5862bac30e57d43a905630274af301a4fa2ab3997e5798c92972821a2e

galaxy.web.base.interactive_environments INFO 2016-01-02 03:32:49,542 Inspecting docker container 81bdfb5862bac30e57d43a905630274af301a4fa2ab3997e5798c92972821a2e
with command [docker inspect 81bdfb5862bac30e57d43a905630274af301a4fa2ab3997e5798c92972821a2e
]
172.17.0.1 - - [02/Jan/2016:03:32:49 +0000] "GET /visualization/show/ipython?dataset_id=f2db41e1fa331b3e HTTP/1.1" 200 - "http://localhost:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 138|app: 0|req: 5/15] 172.17.0.1 () {42 vars in 830 bytes} [Sat Jan  2 03:32:49 2016] GET /visualization/show/ipython?dataset_id=f2db41e1fa331b3e => generated 1366 bytes in 629 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 2)
172.17.0.1 - - [02/Jan/2016:03:32:49 +0000] "GET /plugins/interactive_environments/ipython/static/js/ipython.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/ipython?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 139|app: 0|req: 11/16] 172.17.0.1 () {42 vars in 840 bytes} [Sat Jan  2 03:32:49 2016] GET /plugins/interactive_environments/ipython/static/js//ipython.js => generated 301 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)

Rstudio

galaxy.web.base.interactive_environments INFO 2016-01-02 03:56:45,554 Starting docker container for IE rstudio with command [docker run --sig-proxy=true -e DEBUG=false -e "GALAXY_URL=http:/" -e "NOTEBOOK_USERNAME=galaxy" -e "CORS_ORIGIN=http://localhost:8080" -e "HISTORY_ID=f597429621d6eb2b" -e "NOTEBOOK_PASSWORD=aabntobhgyvxqms0ma5p9i4l" -e "PROXY_PREFIX=" -e "API_KEY=866760228e04dcb00e88ef3c0c82e6a7" -e "REMOTE_HOST=172.17.0.1" -e "USER_EMAIL=m.lorenc@qut.edu.au" -d -P -v "/export/galaxy-central/database/files/tmp59YU5j:/import/"  erasche/docker-rstudio-notebook:15.10]
galaxy.web.base.interactive_environments DEBUG 2016-01-02 03:56:45,979 Container id: c9175e6ab6bad72cfecf603bfca66c73b3d022b95922a66c2c4f24e930f78f07

galaxy.web.base.interactive_environments INFO 2016-01-02 03:56:45,979 Inspecting docker container c9175e6ab6bad72cfecf603bfca66c73b3d022b95922a66c2c4f24e930f78f07
with command [docker inspect c9175e6ab6bad72cfecf603bfca66c73b3d022b95922a66c2c4f24e930f78f07
]
172.17.0.1 - - [02/Jan/2016:03:56:45 +0000] "GET /visualization/show/rstudio?dataset_id=f2db41e1fa331b3e HTTP/1.1" 200 - "http://localhost:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 130|app: 0|req: 13/30] 172.17.0.1 () {42 vars in 830 bytes} [Sat Jan  2 03:56:45 2016] GET /visualization/show/rstudio?dataset_id=f2db41e1fa331b3e => generated 1559 bytes in 567 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 1)
172.17.0.1 - - [02/Jan/2016:03:56:46 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/prng4.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 130|app: 0|req: 14/31] 172.17.0.1 () {42 vars in 850 bytes} [Sat Jan  2 03:56:46 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//prng4.js => generated 306 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 3)
172.17.0.1 - - [02/Jan/2016:03:56:46 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/rng.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 130|app: 0|req: 15/32] 172.17.0.1 () {42 vars in 846 bytes} [Sat Jan  2 03:56:46 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//rng.js => generated 304 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 2)
172.17.0.1 - - [02/Jan/2016:03:56:46 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/rsa.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 130|app: 0|req: 16/33] 172.17.0.1 () {42 vars in 846 bytes} [Sat Jan  2 03:56:46 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//rsa.js => generated 304 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 3)
172.17.0.1 - - [02/Jan/2016:03:56:46 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/jsbn.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 131|app: 0|req: 18/34] 172.17.0.1 () {42 vars in 848 bytes} [Sat Jan  2 03:56:46 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//jsbn.js => generated 305 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
172.17.0.1 - - [02/Jan/2016:03:56:46 +0000] "GET /plugins/interactive_environments/rstudio/static/js/rstudio.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 130|app: 0|req: 17/35] 172.17.0.1 () {42 vars in 840 bytes} [Sat Jan  2 03:56:46 2016] GET /plugins/interactive_environments/rstudio/static/js//rstudio.js => generated 301 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
172.17.0.1 - - [02/Jan/2016:03:56:46 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/base64.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 131|app: 0|req: 19/36] 172.17.0.1 () {42 vars in 852 bytes} [Sat Jan  2 03:56:46 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//base64.js => generated 307 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)

Without -e DOCKER_PARENT=True

docker run -p 8080:80 -p 8021:21 -p 8800:8800 --privileged=true -v /var/run/docker.sock:/var/run/docker.sock -v /home/galaxy/galaxy_storage/:/export/ -e "GALAXY_CONFIG_ADMIN_USERS=m.lorenc@qut.edu.au" -e "host=0.0.0.0" bgruening/galaxy-stable
munged: Warning: Logfile is insecure: group-writable permissions set on "/var/log"
Enable Galaxy Interactive Environments.
ln: failed to create symbolic link ‘/sys/fs/cgroup/systemd/name=systemd’: Operation not permitted
/usr/lib/python2.7/dist-packages/supervisor/options.py:295: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '

...

ipython

galaxy.web.base.interactive_environments INFO 2016-01-02 03:43:54,365 Starting docker container for IE ipython with command [docker run --sig-proxy=true -e DEBUG=false -e "GALAXY_URL=http:/" -e "CORS_ORIGIN=http://localhost:8080" -e "HISTORY_ID=f597429621d6eb2b" -e "NOTEBOOK_PASSWORD=none" -e "DATASET_HID=None" -e "PROXY_PREFIX=" -e "API_KEY=866760228e04dcb00e88ef3c0c82e6a7" -e "REMOTE_HOST=172.17.0.1" -e "USER_EMAIL=m.lorenc@qut.edu.au" -d -P   bgruening/docker-ipython-notebook:15.10.1]
galaxy.web.base.interactive_environments DEBUG 2016-01-02 03:43:54,758 Container id: b09e174dba5fd423c91dd5dea94ac23935c680eb5e4c0f7554256028057da314

galaxy.web.base.interactive_environments INFO 2016-01-02 03:43:54,759 Inspecting docker container b09e174dba5fd423c91dd5dea94ac23935c680eb5e4c0f7554256028057da314
with command [docker inspect b09e174dba5fd423c91dd5dea94ac23935c680eb5e4c0f7554256028057da314
]
172.17.0.1 - - [02/Jan/2016:03:43:54 +0000] "GET /visualization/show/ipython?dataset_id=f2db41e1fa331b3e HTTP/1.1" 200 - "http://localhost:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 188|app: 0|req: 7/15] 172.17.0.1 () {42 vars in 830 bytes} [Sat Jan  2 03:43:54 2016] GET /visualization/show/ipython?dataset_id=f2db41e1fa331b3e => generated 1366 bytes in 536 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 2)
172.17.0.1 - - [02/Jan/2016:03:43:54 +0000] "GET /plugins/interactive_environments/ipython/static/js/ipython.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/ipython?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 187|app: 0|req: 9/16] 172.17.0.1 () {42 vars in 840 bytes} [Sat Jan  2 03:43:54 2016] GET /plugins/interactive_environments/ipython/static/js//ipython.js => generated 301 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)

Rstudio

galaxy.web.base.interactive_environments INFO 2016-01-02 03:45:58,936 Starting docker container for IE rstudio with command [docker run --sig-proxy=true -e DEBUG=false -e "GALAXY_URL=http:/" -e "NOTEBOOK_USERNAME=galaxy" -e "CORS_ORIGIN=http://localhost:8080" -e "HISTORY_ID=f597429621d6eb2b" -e "NOTEBOOK_PASSWORD=84amct3ohj7azw7gu4r819gy" -e "PROXY_PREFIX=" -e "API_KEY=866760228e04dcb00e88ef3c0c82e6a7" -e "REMOTE_HOST=172.17.0.1" -e "USER_EMAIL=m.lorenc@qut.edu.au" -d -P -v "/export/galaxy-central/database/files/tmpH5wmaa:/import/"  erasche/docker-rstudio-notebook:15.10]
galaxy.web.base.interactive_environments DEBUG 2016-01-02 03:45:59,335 Container id: f8620a9648a882218cf3122910b52a76d7ced8a32c215b6e616d3a1340d42669

galaxy.web.base.interactive_environments INFO 2016-01-02 03:45:59,335 Inspecting docker container f8620a9648a882218cf3122910b52a76d7ced8a32c215b6e616d3a1340d42669
with command [docker inspect f8620a9648a882218cf3122910b52a76d7ced8a32c215b6e616d3a1340d42669
]
172.17.0.1 - - [02/Jan/2016:03:45:58 +0000] "GET /visualization/show/rstudio?dataset_id=f2db41e1fa331b3e HTTP/1.1" 200 - "http://localhost:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 187|app: 0|req: 10/17] 172.17.0.1 () {42 vars in 830 bytes} [Sat Jan  2 03:45:58 2016] GET /visualization/show/rstudio?dataset_id=f2db41e1fa331b3e => generated 1559 bytes in 537 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 1)
172.17.0.1 - - [02/Jan/2016:03:45:59 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/prng4.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 187|app: 0|req: 12/18] 172.17.0.1 () {42 vars in 850 bytes} [Sat Jan  2 03:45:59 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//prng4.js => generated 306 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 3)
172.17.0.1 - - [02/Jan/2016:03:45:59 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/rng.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 187|app: 0|req: 12/19] 172.17.0.1 () {42 vars in 846 bytes} [Sat Jan  2 03:45:59 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//rng.js => generated 304 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 2)
172.17.0.1 - - [02/Jan/2016:03:45:59 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/base64.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 187|app: 0|req: 15/20] 172.17.0.1 () {42 vars in 852 bytes} [Sat Jan  2 03:45:59 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//base64.js => generated 307 bytes in 0 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)
172.17.0.1 - - [02/Jan/2016:03:45:59 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/rsa.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
172.17.0.1 - - [02/Jan/2016:03:45:59 +0000] "GET /plugins/interactive_environments/rstudio/static/js/crypto/jsbn.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 187|app: 0|req: 15/21] 172.17.0.1 () {42 vars in 846 bytes} [Sat Jan  2 03:45:59 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//rsa.js => generated 304 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 3)
[pid: 187|app: 0|req: 15/22] 172.17.0.1 () {42 vars in 848 bytes} [Sat Jan  2 03:45:59 2016] GET /plugins/interactive_environments/rstudio/static/js/crypto//jsbn.js => generated 305 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 1)
172.17.0.1 - - [02/Jan/2016:03:45:59 +0000] "GET /plugins/interactive_environments/rstudio/static/js/rstudio.js HTTP/1.1" 404 - "http://localhost:8080/visualization/show/rstudio?dataset_id=f2db41e1fa331b3e" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0"
[pid: 188|app: 0|req: 8/23] 172.17.0.1 () {42 vars in 840 bytes} [Sat Jan  2 03:45:59 2016] GET /plugins/interactive_environments/rstudio/static/js//rstudio.js => generated 301 bytes in 1 msecs (HTTP/1.1 404) 1 headers in 51 bytes (1 switches on core 0)

I logged in to the container and the ipython.ini and rstudio.ini are not enable docker run -it docker-galaxy-stable-tmp

# ls /galaxy-central/config/plugins/interactive_environments/rstudio/config
rstudio.ini.sample  rstudio.xml
# ls /galaxy-central/config/plugins/interactive_environments/ipython/config
ipython.ini.sample  ipython.xml

What did I do wrong?

Thank you in advance.

Mic

bgruening commented 8 years ago

Please read this section https://github.com/bgruening/docker-galaxy-stable#enabling-interactive-environments-in-galaxy

All what you need to run is:

docker run -d -p 8080:80  -p 8800:8800 --privileged=true  bgruening/galaxy-stable

Please note that the first time you run any of these IE is needs to fetch the container. This can take a while, depending on your connection up to an hour. This means you will see a timeout the first time you start an IE.

mictadlo commented 8 years ago

I tried to use the following command:

> docker run -d -p 8080:80 -p 8800:8800 -p 9002:9002 --privileged=true -e "GALAXY_LOGGING=full" bgruening/galaxy-stable6cd70d1d8b10667cf98c62fb67b1593911d69c0c9617802bc01e6f5ab803c356

and received the following docker error massages:

time="2016-01-04T10:31:04.992632146Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-04T10:31:04.992777695Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-04T10:31:06.329986754Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-04T10:31:06.330140404Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-04T10:31:09.000757480Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-04T10:31:09.000911410Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)"

What did I do wrong?

Thank you in advance.

Mic

bgruening commented 8 years ago

This seems to be a general Docker problem with your driver-backend (btrfs). See here for example: http://stackoverflow.com/questions/30669359/docker-with-btrfs-ubuntu

mictadlo commented 8 years ago

I also posted on docker issue ( https://github.com/docker/docker/issues/19065 ) and received the following answer:

   In order to use btrfs, docker storage dir must be on a btrfs partition.

What kind of distribution the IE docker user use?

mictadlo commented 8 years ago

I was able to sovlve the problem with btrfs, but for some reason IPython and Rstudio still does not want to start:

docker run -d -p 8080:80 -p 8800:8800 -p 9002:9002 --privileged=true -e "GALAXY_LOGGING=full" bgruening/galaxy-stable

IPython

l=info msg="POST /v1.19/containers/c7b9b11a8dc2258a257181cb6c8b607a2a3164360f195ac0e350b79da35c2271/start" 
time="2016-01-05T11:43:12.142937612Z" level=info msg="GET /v1.19/containers/c7b9b11a8dc2258a257181cb6c8b607a2a3164360f195ac0e350b79da35c2271/json" 
time="2016-01-05T11:43:18.541834678Z" level=info msg="POST /v1.19/containers/create" 
time="2016-01-05T11:43:18.613284261Z" level=info msg="POST /v1.19/containers/ed96c8a87e4cd33c641a6d9df201a27f46672a36e1dd63c561a67b0d997db2d9/start" 
time="2016-01-05T11:43:19.005219828Z" level=info msg="GET /v1.19/containers/ed96c8a87e4cd33c641a6d9df201a27f46672a36e1dd63c561a67b0d997db2d9/json" 
time="2016-01-05T12:54:02.283844699Z" level=info msg="POST /v1.19/containers/create" 
time="2016-01-05T12:54:02.319168183Z" level=info msg="POST /v1.19/containers/5bd131b0c559266543f7a1cfad4d92353b4cc36d67d28aa7a11ddc0c7493adb7/start" 
time="2016-01-05T12:54:02.719922748Z" level=info msg="GET /v1.19/containers/5bd131b0c559266543f7a1cfad4d92353b4cc36d67d28aa7a11ddc0c7493adb7/json"

Rstudio

l=info msg="POST /v1.19/containers/ed96c8a87e4cd33c641a6d9df201a27f46672a36e1dd63c561a67b0d997db2d9/start" 
time="2016-01-05T11:43:19.005219828Z" level=info msg="GET /v1.19/containers/ed96c8a87e4cd33c641a6d9df201a27f46672a36e1dd63c561a67b0d997db2d9/json" 
time="2016-01-05T12:54:02.283844699Z" level=info msg="POST /v1.19/containers/create" 
time="2016-01-05T12:54:02.319168183Z" level=info msg="POST /v1.19/containers/5bd131b0c559266543f7a1cfad4d92353b4cc36d67d28aa7a11ddc0c7493adb7/start" 
time="2016-01-05T12:54:02.719922748Z" level=info msg="GET /v1.19/containers/5bd131b0c559266543f7a1cfad4d92353b4cc36d67d28aa7a11ddc0c7493adb7/json" 
time="2016-01-05T12:55:42.771146845Z" level=info msg="POST /v1.19/containers/create" 
time="2016-01-05T12:55:42.808802748Z" level=info msg="POST /v1.19/containers/4c40b6c597cae2409bb50e29edb937bc4b99cc8c4909fda65911e4a9a979fe74/start" 
time="2016-01-05T12:55:43.211055828Z" level=info msg="GET /v1.19/containers/4c40b6c597cae2409bb50e29edb937bc4b99cc8c4909fda65911e4a9a979fe74/json"

What did I do wrong that IPython and Rstudio do not want start?

Solution for the btrfs problem:

# Install docker-engine
apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install linux-image-extra-4.2.0-22-generic linux-headers-4.2.0-22-generic btrfs-tools
apt-get purge lxc-docker
apt-cache policy docker-engine
apt-get install docker-engine
reboot

# Setup btrfs partition for docker
service docker stop
rm -rf /var/lib/docker/
mkdir /var/lib/docker/
mkfs.btrfs -f /dev/sda6
echo "/dev/sda6 /var/lib/docker btrfs defaults 0 0" >> /etc/fstab
mount -a
df -hT
service docker start 

#docker without sudo
sudo service docker start
sudo groupadd docker
sudo gpasswd -a lorencm docker
sudo service docker restart
sudo service docker status

To solve the problem ip6 and use nm-tool to find my DNS server (https://robinwinslow.uk/2014/08/27/fix-docker-networking/):

sudo vim /etc/default/docker
uncomment line DOCKER_OPTS="--dns 8.8.8.8 --dns 131.181.127.32"
restart docker engine service docker restart
jjkoehorst commented 8 years ago

I am having a similar issue but I am confused where it goes wrong...

/usr/bin/docker daemon -g /var/lib/docker2

INFO[0000] API listen on /var/run/docker.sock           
INFO[0000] [graphdriver] using prior storage driver "aufs" 
INFO[0000] Firewalld running: false                     
INFO[0000] Default bridge (docker0) is assigned with an IP address 172.17.0.1/16. Daemon option --bip can be used to set a preferred IP address 
WARN[0000] Your kernel does not support swap memory limit. 
INFO[0000] Loading containers: start.                   
..........
INFO[0000] Loading containers: done.                    
INFO[0000] Daemon has completed initialization          
INFO[0000] Docker daemon                                 commit=a34a1d5 execdriver=native-0.2 graphdriver=aufs version=1.9.1

docker run -d -p 8081:80 -p 8800:8800 -p 9002:9002 --privileged=true bgruening/galaxy-stable 1a39119d450b896d900bfb92784c1b3baa3bbf8040580280818374a25ffd1c67

Exited too quickly (process log may have details)   docker  Clear Log Tail -f

)" 
time="2016-01-05T16:31:25.599893173Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-05T16:31:25.599984428Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-05T16:31:28.491223109Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-05T16:31:28.491438544Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-05T16:31:31.830906523Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-05T16:31:31.831125685Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
  Refresh

Now I am getting the btrfs error while I am using ext4 ubuntu 14.04 system using according to docker a prior storage driver "aufs". What am I doing wrong?

Any ideas?

mictadlo commented 8 years ago

You need to create a new partition just for the docker container and format it so it get btrfs. Please read my above post:

    Solution for the btrfs problem

which will solve you problem. However, IE still does not load.

mictadlo commented 8 years ago

I had to change grub setting in the following way

sudo vim /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1"

then

sudo update-grub && sudo reboot

Unfortunately, I received _Running modprobe bridge nfnat failed with message: , error: exit status 1 (https://github.com/docker/docker/issues/19114)

level=info msg="[graphdriver] using prior storage driver \"btrfs\"" 
time="2016-01-06T01:22:38.280066807Z" level=info msg="Listening for HTTP on unix (/var/run/docker.sock)" 
time="2016-01-06T01:22:38.280212283Z" level=warning msg="Running modprobe bridge nf_nat failed with message: , error: exit status 1" 
time="2016-01-06T01:22:38.347388896Z" level=info msg="Loading containers: start." 
....
time="2016-01-06T01:22:38.349652297Z" level=info msg="Loading containers: done." 
time="2016-01-06T01:22:38.349687201Z" level=info msg="Daemon has completed initialization" 
time="2016-01-06T01:22:38.349710376Z" level=info msg="Docker daemon" commit=786b29d execdriver=native-0.2 graphdriver=btrfs version=1.7.1 
time="2016-01-06T01:31:35.025738746Z" level=info msg="POST /v1.19/containers/create" 
time="2016-01-06T01:31:35.079254813Z" level=info msg="POST /v1.19/containers/c26c2046e89b516d32bb12626f3c82c0d26e83fe6d5b60318d683016cb3d08f7/start" 
time="2016-01-06T01:31:35.469049046Z" level=info msg="GET /v1.19/containers/c26c2046e89b516d32bb12626f3c82c0d26e83fe6d5b60318d683016cb3d08f7/json"

Unfortunately, Ipython and Rstudio did not load.

What do I miss?

mictadlo commented 8 years ago

@jjkoehorst were you able to get IE running? Unfortunately, I am not able to get it running.

jjkoehorst commented 8 years ago

No but I have some issues with creating new partitions and grub as it is not my machine it is running on. Not sure if they are willing to allow me to do such things... I hope it can be fixed on the image side as I cannot believe that everybody needs to format or repartition their computer just to get this to work?

mvdbeek commented 8 years ago

I really don't think it's necessary to use btrfs, docker-in-docker used to work in the past. Sorry I can't be more specific, but creating a btrfs partition just for docker is not addressing the problem at its root, and is likely not related to getting IE's to work, and in the worst case scenario you'll be loosing data.

mictadlo commented 8 years ago

Has anyone a VM with working IE? VM might solve the problem with Docker-in-Docker.

mvdbeek commented 8 years ago

OK, I got it (almost) working, but I bet it will once the ipython container finishes downloading. Can you start your docker container, then enter (docker exec -it (id_of_the_container) bash). Edit /etc/supervisor/conf.d/galaxy to look like this:

[program:docker]
directory       = /
command         = docker -d --host=unix:///var/run/docker.sock --host=tcp://0.0.0.0:2375 --storage-driver=vfs
autostart       = False
autorestart     = True
user            = root
startsecs       = 0
redirect_stderr = true

Then supervisorctl update && supervisorctl start docker. Then a docker ps on the commandline should give you this:

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

From there on you should have no problems anymore. I'll make a PR once you confirm it works.

mvdbeek commented 8 years ago

actually, a PR is not necessary, these changes are already in docker-galaxy-stable:dev.

mictadlo commented 8 years ago

I followed your instructions, but Ipython and Rstudio does not appear in the browser. However, docker ps show that the images have been loaded.

root@e8ea7c754692:/galaxy-central# docker images
REPOSITORY                          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
bgruening/docker-ipython-notebook   15.10.1             8ab94b7005ac        3 weeks ago         1.133 GB
erasche/docker-rstudio-notebook     15.10               228f5f993a0d        12 weeks ago        1.19 GB

# docker ps    
CONTAINER ID        IMAGE                                   COMMAND                CREATED             STATUS              PORTS                   NAMES
ff0092b5b46a        erasche/docker-rstudio-notebook:15.10   "/bin/sh -c /startup   52 seconds ago      Up 27 seconds       0.0.0.0:32770->80/tcp   drunk_thompson  

# docker ps
CONTAINER ID        IMAGE                                       COMMAND                CREATED              STATUS              PORTS                     NAMES
06446cd77d87        bgruening/docker-ipython-notebook:15.10.1   "/bin/sh -c /startup   About a minute ago   Up 18 seconds       0.0.0.0:32772->6789/tcp   furious_euclid

What did I do wrong?

mvdbeek commented 8 years ago

For now it doens't work for me either. But there were major changes in release_16.01, so I'm trying now with bgruening/docker-galaxy-stable:dev.

bgruening commented 8 years ago

Hi!

I hope I will find time this weekend to have a deeper look. But btrfs as backend is not needed, at least it was not needed with Ubuntu 15.04 and Docker < 1.8. The problem is that Docker changes to frequently and there are so many backends that do not work properly. This is not only because of Docker but also because of different distributions shipping buggy backends, like aufs under Ubuntu.

For example overlay is now a recommended backend, but with this backend I can not even compile this image under my Linux. Docker > 1.8 changed the Docker in Docker part, independent of the backend and so on ... so I guess the problem here is more complex.

Can you please send me the combination of Docker version, Docker backend, Operating System and Galaxy-Docker version? I will try to set up testing instances if I find time.

I have some code to test Docker in Docker on travis, but this only test it on travis Docker backend, but certainly, better than nothing.

Sorry for the trouble. It's still working for me, but I haven't updated my system since month ...

mictadlo commented 8 years ago

Thank you for trying to fix it. I am using Linux Mint 17.3 (kde 64 bit) on a VirtualBox, docker-engine 1.9.1, Kernel 4.2.0-23. Please find below the outputs:

Software information

> uname -a
Linux galaxy-VirtualBox 4.2.0-23-generic #28~14.04.1-Ubuntu SMP Thu Dec 31 13:40:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

. 

> docker info
Containers: 11
Images: 108
Server Version: 1.9.1
Storage Driver: btrfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.2.0-23-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 4
Total Memory: 39.79 GiB
Name: galaxy-VirtualBox
ID: 7YSR:CDFW:PTC2:N2JB:RNX7:32TE:PNBX:W477:ECJX:AK45:4RFI:UGJT

.

> docker version
Client:
Version:      1.9.1
API version:  1.21
Go version:   go1.4.2
Git commit:   a34a1d5
Built:        Fri Nov 20 13:12:04 UTC 2015
OS/Arch:      linux/amd64

Server:
Version:      1.9.1
API version:  1.21
Go version:   go1.4.2
Git commit:   a34a1d5
Built:        Fri Nov 20 13:12:04 UTC 2015
OS/Arch:      linux/amd64

.

> mount
/dev/sda1 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/cgroup type tmpfs (rw)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)
none on /sys/fs/pstore type pstore (rw)
tracefs on /var/lib/ureadahead/debugfs/tracing type tracefs (rw,relatime)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu type cgroup (rw,relatime,cpu)
cgroup on /sys/fs/cgroup/cpuacct type cgroup (rw,relatime,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,relatime,net_cls)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,relatime,perf_event)
cgroup on /sys/fs/cgroup/net_prio type cgroup (rw,relatime,net_prio)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,relatime,hugetlb)
/dev/sda6 on /var/lib/docker type btrfs (rw)
systemd on /sys/fs/cgroup/systemd type cgroup (rw,noexec,nosuid,nodev,none,name=systemd)

.

> bash Downloads/check-config.sh 
warning: /proc/config.gz does not exist, searching other paths for kernel config ...
info: reading kernel config from /boot/config-4.2.0-23-generic ...

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- apparmor: enabled and tools installed
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_DEVPTS_MULTIPLE_INSTANCES: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_MACVLAN: enabled (as module)
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled (as module)
- CONFIG_BRIDGE_NETFILTER: enabled (as module)
- CONFIG_NF_NAT_IPV4: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_NF_NAT_NEEDED: enabled
- CONFIG_POSIX_MQUEUE: enabled

Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_MEMCG_KMEM: enabled
- CONFIG_MEMCG_SWAP: enabled
- CONFIG_MEMCG_SWAP_ENABLED: missing
(note that cgroup swap accounting is not enabled in your kernel config, you can enable it by setting boot option "swapaccount=1")
- CONFIG_BLK_CGROUP: enabled
- CONFIG_IOSCHED_CFQ: enabled
- CONFIG_BLK_DEV_THROTTLING: enabled
- CONFIG_CGROUP_PERF: enabled
- CONFIG_CGROUP_HUGETLB: enabled
- CONFIG_NET_CLS_CGROUP: enabled (as module)
- CONFIG_CGROUP_NET_PRIO: enabled
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: missing
- CONFIG_EXT3_FS: missing
- CONFIG_EXT3_FS_XATTR: missing
- CONFIG_EXT3_FS_POSIX_ACL: missing
- CONFIG_EXT3_FS_SECURITY: missing
(enable these ext3 configs if you are using ext3 as backing filesystem)
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Storage Drivers:
  - "aufs":
- CONFIG_AUFS_FS: enabled (as module)
  - "btrfs":
- CONFIG_BTRFS_FS: enabled (as module)
  - "devicemapper":
- CONFIG_BLK_DEV_DM: enabled
- CONFIG_DM_THIN_PROVISIONING: enabled (as module)
  - "overlay":
- CONFIG_OVERLAY_FS: enabled (as module)
  - "zfs":
- /dev/zfs: missing
- zfs command: available
- zpool command: available

How to set it up the sotfware

Install docker-engine

    wget -qO- https://get.docker.com/ | sh
    apt-get update
    apt-get install linux-image-extra-4.2.0-23-generic linux-headers-4.2.0-23-generic btrfs-tools zfs-fuse aufs-tools cgroup-lite

Fix for "Your kernel does not support swap memory limit."

/etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1"

then sudo update-grub && sudo reboot

Setup btrfs partition for docker

service docker stop
rm -rf /var/lib/docker/
mkdir /var/lib/docker/
mkfs.btrfs -f /dev/sda6
echo "/dev/sda6 /var/lib/docker btrfs defaults 0 0" >> /etc/fstab
mount -a
df -hT
service docker start

docker without sudo

sudo service docker start
sudo groupadd docker
sudo gpasswd -a lorencm docker
sudo service docker restart
sudo service docker status

To solve the problem ip6 and use nm-tool to find my DNS server (https://robinwinslow.uk/2014/08/27/fix-docker-networking/):

sudo vim /etc/default/docker
uncomment line DOCKER_OPTS="--dns 8.8.8.8 --dns 131.181.127.32"
restart docker engine service docker restart
bgruening commented 8 years ago

Ok, here a fist status report. I updated the Ubuntu box to 15.10, which is exactly your kernel.

 % uname -a
Linux bag 4.2.0-23-generic #28-Ubuntu SMP Sun Dec 27 17:47:31 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

and voila, Docker in Docker is working anymore. Using the overlay backend I can not build the image, reported here: https://github.com/docker/docker/issues/12327

This means that you really need a recent, recent kernel to get overlay working: https://lkml.org/lkml/2015/10/9/620

Using the aufs backend, kicksoff Docker in Docker and pulling IPython keeps forever and fills my disc space with 50 GB and never ends. This is reproducibly with Docker 1.6 and Docker 1.9 so I conclude this is a Ubuntu problem for now. Not sure what we con do from this point on.

@mvdbeek that the Docker daemon does not start is dues to a change in Docker 1.9 I have a potential fix in https://github.com/bgruening/docker-galaxy-stable/tree/more_travis. But this does only work with Docker > 1.9 afaik, travis is running Docker 1.8 and still keeps failing.

Unfortunately, no good news here for Ubuntu users :(

mvdbeek commented 8 years ago

@bgruening have you tried with --storage-driver=vfs inside docker? This is working for me (also ubuntu-15.10). Are you sure setting autostart=true for the docker daemon doesn't solve the problem?

bgruening commented 8 years ago

Storage driver is always vfs: https://github.com/galaxyproject/ansible-galaxy-extras/blob/master/templates/supervisor.conf.j2#L169

What do you mean with autostart=true? supervisord? Docker is running this is not the problem, the problems seems to be some wired aufs bug on the host.

So if it's working for you can you try the more_travis branch and see if the container starts from the automatically? Which Docker version are you using?

This gets wired.

mvdbeek commented 8 years ago

Storage driver is always vfs

yes, for dev! but here me (and I think @mictadlo too) were using stable. Switching to dev solved the storage driver btrfs things for me. I will try the more_travis branch.

mvdbeek commented 8 years ago

@bgruening great, with the more_travis branch docker starts up automatically (Docker version 1.9.0, build 76d6bc9).

bgruening commented 8 years ago

And now please tell me that it works for you ;) Are you running the latest kernel - 4.2.0-23-generic?

mvdbeek commented 8 years ago

nope, but I can update:

Linux iMac-ubuntu 4.2.0-21-generic
bgruening commented 8 years ago

Maybe nothing is working than for you as for me ;)

mvdbeek commented 8 years ago

nope, everything still good, as far as I can tell.

(.venv)marius@iMac-ubuntu:~$ docker run --privileged more_travis
Traceback (most recent call last):
  File "/usr/bin/ansible", line 36, in <module>
    from ansible.runner import Runner
ImportError: No module named ansible.runner
Enable Galaxy Interactive Environments.
Starting ProFTP
proftpd: started
Starting Galaxy reports webapp
reports: started
Starting nodejs
galaxy:galaxy_nodejs_proxy: ERROR (abnormal termination)
Starting slurmctld
Starting slurmd
munged: Warning: Logfile is insecure: group-writable permissions set on "/var/log"
docker: started
==> /home/galaxy/handler0.log <==
galaxy.jobs.runners.drmaa INFO 2016-01-10 12:10:09,092 Overriding DRMAA_LIBRARY_PATH due to runner plugin parameter: /usr/lib/slurm-drmaa/lib/libdrmaa.so
galaxy.jobs.runners DEBUG 2016-01-10 12:10:09,100 Starting 2 SlurmRunner workers
galaxy.jobs DEBUG 2016-01-10 12:10:09,100 Loaded job runner 'galaxy.jobs.runners.slurm:SlurmJobRunner' as 'slurm'
galaxy.jobs.handler DEBUG 2016-01-10 12:10:09,100 Loaded job runners plugins: slurm
galaxy.jobs.handler INFO 2016-01-10 12:10:09,101 job handler stop queue started
galaxy.jobs.handler INFO 2016-01-10 12:10:09,113 job handler queue started
galaxy.sample_tracking.external_service_types DEBUG 2016-01-10 12:10:09,115 Loaded external_service_type: Simple unknown sequencer 1.0.0
galaxy.sample_tracking.external_service_types DEBUG 2016-01-10 12:10:09,117 Loaded external_service_type: Applied Biosystems SOLiD 1.0.0
galaxy.workflow.scheduling_manager DEBUG 2016-01-10 12:10:09,130 Starting workflow schedulers
galaxy.queue_worker INFO 2016-01-10 12:10:09,132 Binding and starting galaxy control worker for handler0

==> /home/galaxy/handler1.log <==
galaxy.jobs.runners.drmaa INFO 2016-01-10 12:10:10,598 Overriding DRMAA_LIBRARY_PATH due to runner plugin parameter: /usr/lib/slurm-drmaa/lib/libdrmaa.so
galaxy.jobs.runners DEBUG 2016-01-10 12:10:10,602 Starting 2 SlurmRunner workers
galaxy.jobs DEBUG 2016-01-10 12:10:10,602 Loaded job runner 'galaxy.jobs.runners.slurm:SlurmJobRunner' as 'slurm'
galaxy.jobs.handler DEBUG 2016-01-10 12:10:10,603 Loaded job runners plugins: slurm
galaxy.jobs.handler INFO 2016-01-10 12:10:10,603 job handler stop queue started
galaxy.jobs.handler INFO 2016-01-10 12:10:10,614 job handler queue started
galaxy.sample_tracking.external_service_types DEBUG 2016-01-10 12:10:10,615 Loaded external_service_type: Simple unknown sequencer 1.0.0
galaxy.sample_tracking.external_service_types DEBUG 2016-01-10 12:10:10,616 Loaded external_service_type: Applied Biosystems SOLiD 1.0.0
galaxy.workflow.scheduling_manager DEBUG 2016-01-10 12:10:10,621 Starting workflow schedulers
galaxy.queue_worker INFO 2016-01-10 12:10:10,625 Binding and starting galaxy control worker for handler1

==> /home/galaxy/reports.log <==
galaxy.web.framework.base DEBUG 2016-01-10 12:09:44,371 Enabling 'sample_tracking' controller, class: SampleTracking
galaxy.web.framework.base DEBUG 2016-01-10 12:09:44,372 Enabling 'root' controller, class: Report
galaxy.web.framework.base DEBUG 2016-01-10 12:09:44,374 Enabling 'users' controller, class: Users
galaxy.webapps.reports.buildapp DEBUG 2016-01-10 12:09:44,375 Enabling 'httpexceptions' middleware
galaxy.webapps.reports.buildapp DEBUG 2016-01-10 12:09:44,376 Enabling 'recursive' middleware
galaxy.webapps.reports.buildapp DEBUG 2016-01-10 12:09:44,383 Enabling 'error' middleware
galaxy.webapps.reports.buildapp DEBUG 2016-01-10 12:09:44,384 Enabling 'trans logger' middleware
galaxy.webapps.reports.buildapp DEBUG 2016-01-10 12:09:44,385 Enabling 'x-forwarded-host' middleware
Starting server in PID 96.
serving on http://127.0.0.1:9001

==> /home/galaxy/slurmctld.log <==
[2016-01-10T12:10:08.188] error: Could not open trigger state file /tmp/slurm/trigger_state: No such file or directory
[2016-01-10T12:10:08.188] error: NOTE: Trying backup state save file. Triggers may be lost!
[2016-01-10T12:10:08.188] No trigger state file (/tmp/slurm/trigger_state.old) to recover
[2016-01-10T12:10:08.188] error: Incomplete trigger data checkpoint file
[2016-01-10T12:10:08.188] read_slurm_conf: backup_controller not specified.
[2016-01-10T12:10:08.188] Reinitializing job accounting state
[2016-01-10T12:10:08.188] cons_res: select_p_reconfigure
[2016-01-10T12:10:08.188] cons_res: select_p_node_init
[2016-01-10T12:10:08.188] cons_res: preparing for 1 partitions
[2016-01-10T12:10:08.188] Running as primary controller

==> /home/galaxy/slurmd.log <==
[2016-01-10T12:10:08.186] Node configuration differs from hardware: CPUs=4:4(hw) Boards=1:1(hw) SocketsPerBoard=4:1(hw) CoresPerSocket=1:4(hw) ThreadsPerCore=1:1(hw)
[2016-01-10T12:10:08.186] Gathering cpu frequency information for 4 cpus
[2016-01-10T12:10:08.188] slurmd version 2.6.5 started
[2016-01-10T12:10:08.188] slurmd started on Sun, 10 Jan 2016 12:10:08 +0000
[2016-01-10T12:10:08.200] CPUs=4 Boards=1 Sockets=4 Cores=1 Threads=1 Memory=16024 TmpDisk=8012 Uptime=301

==> /home/galaxy/uwsgi.log <==
galaxy.webapps.galaxy.buildapp DEBUG 2016-01-10 12:10:10,144 Enabling 'x-forwarded-host' middleware
galaxy.webapps.galaxy.buildapp DEBUG 2016-01-10 12:10:10,144 Enabling 'Request ID' middleware
Sun Jan 10 12:10:10 2016 - WSGI app 0 (mountpoint='') ready in 33 seconds on interpreter 0x193f280 pid: 52 (default app)
Sun Jan 10 12:10:10 2016 - *** uWSGI is running in multiple interpreter mode ***
Sun Jan 10 12:10:10 2016 - spawned uWSGI master process (pid: 52)
Sun Jan 10 12:10:10 2016 - spawned uWSGI worker 1 (pid: 235, cores: 4)
galaxy.queue_worker INFO 2016-01-10 12:10:10,148 Binding and starting galaxy control worker for main.1
Sun Jan 10 12:10:10 2016 - spawned uWSGI worker 2 (pid: 237, cores: 4)
Sun Jan 10 12:10:10 2016 - *** Stats server enabled on 127.0.0.1:9191 fd: 11 ***
galaxy.queue_worker INFO 2016-01-10 12:10:10,150 Binding and starting galaxy control worker for main.2
(.venv)marius@iMac-ubuntu:~$ docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                                                                                                      NAMES
1c0b74eebf1e        more_travis           "/usr/bin/startup"       48 seconds ago      Up 47 seconds       21/tcp, 80/tcp, 8800/tcp, 9002/tcp                                                                                         gigantic_pare
(.venv)marius@iMac-ubuntu:~$ docker exec -it 1c0b74eebf1e bash
root@1c0b74eebf1e:/galaxy-central# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
root@1c0b74eebf1e:/galaxy-central# docker pull ubuntu:14.04
14.04: Pulling from library/ubuntu
fcee8bcfe180: Pull complete 
4cdc0cbc1936: Pull complete 
d9e545b90db8: Pull complete 
c4bea91afef3: Pull complete 
Digest: sha256:098d121c6a9b39080f835563695f8e05faf765f46c174570e61d08197e82b820
Status: Downloaded newer image for ubuntu:14.04
root@1c0b74eebf1e:/galaxy-central# docker run -it ubuntu:14.04 bash
root@b6ad56941d0c:/# 
(.venv)marius@iMac-ubuntu:~$ uname -a
Linux iMac-ubuntu 4.2.0-23-generic #28-Ubuntu SMP Sun Dec 27 17:47:31 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
(.venv)marius@iMac-ubuntu:~$ docker --version
Docker version 1.9.1, build a34a1d5
mvdbeek commented 8 years ago

I haven't tested the IEs though, I'll do that next.

bgruening commented 8 years ago

@mvdbeek travis is now happy and is testing at least if we can run Docker in Docker. But this is bound to one Docker version. Currently we test against 1.9.1.

bgruening commented 8 years ago

So end of the weekend: If you take this branch https://github.com/galaxyproject/ansible-galaxy-extras/pull/38 and the https://github.com/bgruening/docker-galaxy-stable/tree/more_travis it should hopefully work as expected. Tested with Docker 1.9.1 and aufs backend.

bgruening commented 8 years ago

@mictadlo @jjkoehorst @mvdbeek please give this image a try:

docker pull quay.io/bgruening/galaxy:more_travis

If this works it will become Galaxy 16.01 at some point :)

mictadlo commented 8 years ago

At the moment I get Error getting v2 registry: Get https://registry-1.docker.io/v2

time="2016-01-12T05:41:03.769065721Z" level=info msg="POST /v1.21/containers/create" 
time="2016-01-12T05:41:03.770290104Z" level=error msg="Handler for POST /v1.21/containers/create returned error: No such image: bgruening/docker-jupyter-notebook:16.01" 
time="2016-01-12T05:41:03.770317182Z" level=error msg="HTTP Error" err="No such image: bgruening/docker-jupyter-notebook:16.01" statusCode=404 
time="2016-01-12T05:41:03.770890583Z" level=info msg="POST /v1.21/images/create?fromImage=bgruening%2Fdocker-jupyter-notebook&tag=16.01" 
time="2016-01-12T05:41:18.772210683Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection" 
time="2016-01-12T05:42:57.849305053Z" level=info msg="GET /v1.21/images/json" 
time="2016-01-12T05:43:14.054664595Z" level=info msg="POST /v1.21/containers/create" 
time="2016-01-12T05:43:14.056030061Z" level=error msg="Handler for POST /v1.21/containers/create returned error: No such image: erasche/docker-rstudio-notebook:15.10" 
time="2016-01-12T05:43:14.056056783Z" level=error msg="HTTP Error" err="No such image: erasche/docker-rstudio-notebook:15.10" statusCode=404 
time="2016-01-12T05:43:14.056764242Z" level=info msg="POST /v1.21/images/create?fromImage=erasche%2Fdocker-rstudio-notebook&tag=15.10" 
time="2016-01-12T05:43:29.057804625Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection"
bgruening commented 8 years ago

Please keep trying :) Also please with the older ipython. Dockerhub is still migrating from old to new registry, maybe there was a down-time?

mvdbeek commented 8 years ago

with the pre-made docker image i'm back to

root@f15f7701cc7e:/galaxy-central# cat /var/log/supervisor/docker-std*    
time="2016-01-12T10:10:26.041089059Z" level=warning msg="/!\\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\" 
time="2016-01-12T10:10:26.041581670Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T10:10:26.041645243Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T10:10:27.089056699Z" level=warning msg="/!\\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\" 
time="2016-01-12T10:10:27.089475167Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T10:10:27.089533563Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T10:10:29.133579743Z" level=warning msg="/!\\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\" 
time="2016-01-12T10:10:29.133979810Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T10:10:29.134034555Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T10:10:32.494394614Z" level=warning msg="/!\\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\" 
time="2016-01-12T10:10:32.494877992Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T10:10:32.494947116Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)"
bgruening commented 8 years ago

Are you using btrfs on your host as backend? Can you check with aufs?

mvdbeek commented 8 years ago

No btrfs whatsoever, I modified the running container to storage-drver=vfs now and it seems to pull the jupyter image at least. I'll try aufs next.

mvdbeek commented 8 years ago

What is weird is that I build the image from your repo with changeset id 13051d25b0c170afd9d28800c980b669ad16233c and it worked. So in between there was only the update to ansible-galaxy-extras ...

bgruening commented 8 years ago

vfs is suboptimal as it uses a large amount of disc-space. The ansible update removed the hardcoded -s vfs from Docker start, but I though you also have tested the latest image with this latest ansible role?

It strange that it works for me and it does not try to start the btrfs backend. Maybe we should hard-code the aufs backend to be sure.

bgruening commented 8 years ago

Travis is also fine with this change, obviously and they are running aufs afaik.

mvdbeek commented 8 years ago

I added storage-driver=aufs, so far so good, it's pulling the image.

mvdbeek commented 8 years ago

Okay, it works with aufs, I guess hard-coding for the time being is a good idea.

mictadlo commented 8 years ago

I noticed that pre_postgresql has status exited and maybe therefore I still get "Error getting v2 registry: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection".

mvdbeek commented 8 years ago

It's supposed to be "exited", this is unrelated (but could be better.)

mictadlo commented 8 years ago

Is my command correct _docker run -d -p 8080:80 -p 8800:8800 -p 9002:9002 --privileged=true -e "GALAXY_LOGGING=full" -v /var/lib/docker/galaxy_storage02/:/export/ quay.io/bgruening/galaxy:moretravis?

mvdbeek commented 8 years ago

in principle yes, but try without a volume mount, in case something weird happens there.

mictadlo commented 8 years ago

I did not mount the volume, but still received "Error getting v2 registry: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection".

Next, I created another VM with the only difference that docker is on EXT4 and not any more BTRFS.

However, new problems appeared:

> sudo docker daemon --storage-driver=aufs[sudo] password for galaxy: 
INFO[0000] API listen on /var/run/docker.sock           
INFO[0000] Firewalld running: false                     
INFO[0000] Default bridge (docker0) is assigned with an IP address 172.17.0.1/16. Daemon option --bip can be used to set a preferred IP address 
INFO[0000] Loading containers: start.                   
......
INFO[0000] Loading containers: done.                    
INFO[0000] Daemon has completed initialization          
INFO[0000] Docker daemon                                 commit=a34a1d5 execdriver=native-0.2 graphdriver=aufs version=1.9.1
INFO[0019] POST /v1.21/containers/create                
INFO[0019] POST /v1.21/containers/fffd032ba5d5d2515797335095315f5d9ec03405bea98a4d0524f296fad0f933/start 
INFO[0019] No non-localhost DNS nameservers are left in resolv.conf. Using default external servers : [nameserver 8.8.8.8 nameserver 8.8.4.4] 
INFO[0019] IPv6 enabled; Adding default IPv6 external servers : [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]

Inside the docker image:

(wrong filesystem?)" 
time="2016-01-12T23:42:17.759607969Z" level=warning msg="/!\\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\" 
time="2016-01-12T23:42:17.760679367Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T23:42:17.760807664Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T23:42:21.010842670Z" level=warning msg="/!\\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\\" 
time="2016-01-12T23:42:21.011873424Z" level=error msg="[graphdriver] prior storage driver \"btrfs\" failed: prerequisites for driver not satisfied (wrong filesystem?)" 
time="2016-01-12T23:42:21.012185103Z" level=fatal msg="Error starting daemon: error initializing graphdriver: prerequisites for driver not satisfied (wrong filesystem?)"

What did I do wrong?