search

bgruening / docker-galaxy

:whale::bar_chart::books: Docker Images tracking the stable Galaxy releases.
http://bgruening.github.io/docker-galaxy
MIT License
226 stars 134 forks source link

19.01 release shed_tools broken link #485

Closed innovate-invent closed 3 days ago

innovate-invent commented 5 years ago

It appears shed_tools has moved from the /export/shed_tools to /export/galaxy-central/database/shed_tools. This has left an empty and misleading shed_tools folder in /export.

bgruening commented 5 years ago

Mh, can you please try the quay.io/bgruening/galaxy and let me know if you see the same behavior?

innovate-invent commented 5 years ago

quay.io/bgruening/galaxy:latest or quay.io/bgruening/galaxy:19.01?

bgruening commented 5 years ago

Should be both the same hopefully. Let me know if both work ;)

innovate-invent commented 5 years ago

quay.io/bgruening/galaxy:19.01 still has this problem. Should I actually try :latest?

bgruening commented 5 years ago

Then I think I don't understand the problem. After installation of tools I do see this.

(base) root@b53ae30ff741:/galaxy-central# ls -l /export/shed_tools/toolshed.g2.bx.psu.edu/
total 4
drwxr-xr-x 3 galaxy galaxy 4096 Mar  1 17:42 repos

That is what is expected, isn't it?

innovate-invent commented 5 years ago

That is what is expected. What storage driver are you using?

bgruening commented 5 years ago

This has nothing to do with storage driver. How do you start the container. Try without /export or use a fresh /export.

innovate-invent commented 5 years ago

Using a fresh export every time.

docker run -d --rm \
-p 9080:80 \
-p 9021:21 \
-p 9022:22 \
--name Galaxy \
--privileged=true \
-v `pwd`/galaxy-export:/export/ \
-v `pwd`/galaxy-tools:/local_tools \
-e "GALAXY_RELEASE=release_19.01" \
-e "GALAXY_CONFIG_ADMIN_USERS=nolan_w@sfu.ca" \
-e "GALAXY_CONFIG_MASTER_API_KEY=83D5jaba7330aDAAkakjGa247" \
-e "GALAXY_CONFIG_BRAND='Galaxy Dev Test'" \
-e "GALAXY_CONFIG_TOOL_CONFIG_FILE=config/tool_conf.xml.sample,config/shed_tool_conf.xml.sample,/local_tools/tool_conf.xml" \
-e "GALAXY_CONFIG_WATCH_TOOLS=true" \
-e "GALAXY_CONFIG_CONDA_AUTO_INSTALL=true" \
-e "ENABLE_TTS_INSTALL=True" \
quay.io/bgruening/galaxy:19.01
bgruening commented 5 years ago

I'm not sure what's going on here. A few things that might help to debug this.

Running a Galaxy flavor with 19.01 (docker run -i -t --rm quay.io/bgruening/galaxy-ngs-preprocessing:19.01 bash) yields the following which looks correct to me.

Without running startup:

(base) root@45062ebf34f1:/galaxy-central# ls -l /shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/
total 8
drwxrwxrwx 3 root root 4096 Feb 28 21:20 bedtools
drwxrwxrwx 3 root root 4096 Feb 28 21:10 multiqc

After running startup:

(base) root@a736a14dc47b:/galaxy-central# ls -l /export/shed_tools/
total 4
drwxr-xr-x 3 galaxy galaxy 4096 Feb 28 21:09 toolshed.g2.bx.psu.edu

The location where this path is configured is stored in a file called shed_tool_conf.xml and it should look like that:

(base) root@a736a14dc47b:/galaxy-central# head config/shed_tool_conf.xml
<?xml version="1.0" ?>
<toolbox tool_path="../shed_tools">

Can you make sure you have the same tool_path in there?

innovate-invent commented 5 years ago 
docker rmi quay.io/bgruening/galaxy:19.01
docker run -d --rm -p 9080:80 -v /tmp/galaxytmp:/export/ quay.io/bgruening/galaxy:19.01

Deleting the image and re-downloading seems to have resolved this issue.

innovate-invent commented 5 years ago

Scratch that, something after starting the container seems to break the link. I started quay.io/bgruening/galaxy:dev, installed some tools via the beta "Install tools" menu. Now bashing into the container has /export/shed_tools empty. /shed_tools links to /export/shed_tools. /export/galaxy-central/database/shed_tools/ has the installed tools.

bgruening commented 5 years ago

The dev containers is for good reason called dev :)

Its really WIP and has a lot of breaking changes this release: https://github.com/bgruening/docker-galaxy-stable/pull/486

I do not recommend using it atm.

innovate-invent commented 5 years ago

Yea, I need https://github.com/galaxyproject/galaxy/pull/7435 though. It is the same issue between dev and 19.01, I don't think dev is causing it.

bgruening commented 5 years ago

Is 19.01 working for you? I tried to explain what is causing it in https://github.com/bgruening/docker-galaxy-stable/issues/485#issuecomment-469019957 ... 19.01 Docker has a configuration switch to make it work. The 'dev' version of Docker is really different in a lot of aspect and I need to check if this configuration change is still working. You can also check it and change it following https://github.com/bgruening/docker-galaxy-stable/issues/485#issuecomment-469019957 (config/shed_tool_conf.xml)

innovate-invent commented 5 years ago

Sorry, I should have noted that everything you mentioned above was working as expected when I closed the issue. I will switch back to 19.01 and see what happens.

innovate-invent commented 5 years ago

I just started a new instance of 19.01 and /galaxy-central/database/shed_tools is missing until I install a tool.

bgruening commented 3 days ago

Thanks to @jyotipm29 we are back on track :smile: The new 24.1 image contains a lot of changes and reflects the latest developments in Galaxy. I would like to close this PR, but please feel free to reopen and rebase against the latest version.

Please give it a try:

docker run -p 8080:80 -p 8021:21 -p 4002:4002 --privileged=true -e "GALAXY_DESTINATIONS_DEFAULT=slurm_cluster_docker"   -v /tmp/galaxy-data/:/export/ quay.io/bgruening/galaxy:24.1

... or any other combination. The readme has been updated. Please add any useful tip to it.

For a list of changes, see the Changelog.