Custom Interactive Environment Static Files not being moved to /galaxy-central/static/plugins/i_e

[monprin]

I have been trying to use the docker container to develop custom interactive environments. I have a plugin written but have found that while the container does get started, the static files get 404s when the browser tries to fetch them because they are not being copied or linked in the /galaxy-central/static/plugins/interactive_environments folder which seems to be the only place nginx pulls static file from. Is there a configuration parameter or way to start the container such that it automatically configures the custom IEs to work out of the box?

As it stands now I can get it to work with an admittedly pretty ugly hack: running the included plugin_staging.py script after /usr/bin/startup runs, this does work and I have confirmed that the plugin works properly after doing this.

For reference, I called the plugin shiny, but all it is is a renamed copy of the jupyter interactive environment to ensure I was dealing with something that already worked.

Here is the nginx error:

==> /var/log/nginx/error.log <==
2018/12/14 16:51:44 [error] 129#0: *5 open() "/galaxy-central/static/plugins/interactive_environments/shiny/static/js/shiny.js" failed (2: No such file or directory), client: 10.8.16.126, server: , request: "GET /static/plugins/interactive_environments/shiny/static/js/shiny.js?v=1544806220 HTTP/1.1", host: "10.15.33.17:2780", referrer: "http://10.15.33.17:2780/plugins/interactive_environments/shiny/show?image_tag=quay.io%2Fbgruening%2Fdocker-jupyter-notebook%3A17.09&dataset_id=f2db41e1fa331b3e&additional_dataset_ids="

Here is the command to start the container (I have also done it with Docker in Docker, to the same results):

docker run -it -p 2780:80 -p 2721:21 -p 8800:8800 -p 2702:9002 \
    --privileged=true -e "GALAXY_LOGGING=full" -e "DOCKER_PARENT=True" \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /ldata1/home/jeffersj/galaxy-storage/:/export/ \
    bgruening/galaxy-stable

If you need any other details I would be happy to provide them.

Thank you.

[bgruening]

@monprin which version of Galaxy or the container are you using? Galaxy has gone through a refactoring of its client build system and in recent versions. 18.09 might fix it - maybe you can try quay.io/bgruening/galaxy:dev ...?

[monprin]

I tried this last week with the dev version to the same result (with a fresh build of the export folder and freshly renamed IE) and I just did a fresh pull this morning and had the same result. The included Jupyter instance does work.

For reference, this is the docker run command I am using to get it working now for the time being:

bgruening/galaxy-stable(:dev) /bin/bash -c "rm -rf /galaxy-central/config && cp -a /export/galaxy-central/config /galaxy-central/config && python3 /galaxy-central/scripts/plugin_staging.py && /usr/bin/startup"

[bgruening]

Ok, can you please try the following. Jump into the container -i -t and execute make client-production. Let me know if this solves your problem.

[monprin]

I tried it both before and after /usr/bin/startup had run with the dev image (in fresh containers), and neither way worked.

[dannon]

As you've noticed, this isn't bundled into 'make client*' endpoints, but rather the startup staging script you found (https://github.com/galaxyproject/galaxy/blob/dev/scripts/plugin_staging.py), which is executed as a part of common_startup.sh. In the docker deployment of Galaxy, since I think common_startup is skipped, this just doesn't run automatically.

This makes me wonder if anything else in common_startup.sh is maybe missing from the container, since we do expect that to be run?

In any event, @monprin, that's the correct script that will stage your plugins. You should only have to run it on plugin install/update, or if you create a new container.

[bgruening]

This makes me wonder if anything else in common_startup.sh is maybe missing from the container, since we do expect that to be run?

@dannon we run common_startup during container build. But this issue is about local changes, a new IE, and how this is configured. Running common_startup could be run after any manual changes to the plugin directory.

@monprin can you confirm that this works? We could add something to the readme file to make this more obvious.

[monprin]

Happy New Year!

@bgruening When running common_startup.sh before /usr/bin/startup, it does not work, because the plugin staging relies on the config folder movements that are done in /usr/bin/startup.

It also doesn't really make sense to run common_startup.sh after the app has already started given that a lot of what it does has to do with the virtualenv and those changes wouldn't necessarily be reflected in the running app.

Let me know if there is some other way you want me to test it out, such as inserting common_startup.sh somewhere in /usr/bin startup.

Thanks

[bgruening]

@monprin happy new your to you too.

You are correct. But can you try to run ./scripts/plugin_staging.py after your IE modifications. Does this work? We could include this script into the startup script I guess.

[monprin]

Yes that does work, with the caveat that it is carried out after the config folder moving. When I am running it now, I am moving the config folder manually when using run like this:

rm -rf /galaxy-central/config && \
cp -a /export/galaxy-central/config /galaxy-central/config && \
python3 /galaxy-central/scripts/plugin_staging.py && \
/usr/bin/startup"

[bgruening]

Could you simply symlink?

[monprin]

Yeah I believe so, this was just a matter of simplicity.

I tested running ./scripts/plugin_staging.py after /usr/bin/startup had started everything up and it worked correctly, and I believe that does symlinking.

[bgruening]

Cool, so all we need to do is adding this to the startup script, right?

[monprin]

Yeah that should work I think.

[bgruening]

@monprin last question do you need this in 18.09 or can you wait for 19.01?

[bgruening]

Let me know what you think: https://github.com/galaxyproject/ansible-galaxy-extras/compare/18.09...plugin_staging?quick_pull=1

[monprin]

I would prefer 18.09, but if that is inconvenient it isn't that big of a deal for me since the script I have has this baked in now.

The command looks good, but will it get executed if the tail commands above it are set to follow since I believe that blocks further execution until exited?

[zhixuqiu]

I would prefer 18.09, but if that is inconvenient it isn't that big of a deal for me since the script I have has this baked in now.

Yes that does work, with the caveat that it is carried out after the config folder moving. When I am running it now, I am moving the config folder manually when using run like this:

rm -rf /galaxy-central/config && \
cp -a /export/galaxy-central/config /galaxy-central/config && \
python3 /galaxy-central/scripts/plugin_staging.py && \
/usr/bin/startup"

Hello, I want to use a galaxy docker container to develop custom interactive environments with my shiny plugin. But I don't know how to do it. Could you share me a successful instance which include a galaxy docker , plugin set and a shiny docker? best.

[monprin]

Hey @zhixuqiu,

This isn't exactly the correct place for this kind of request. Try heading over to https://galaxyproject.org/support/ for more information on how to troubleshoot issues you are having with your current setup, and if you still need more help, try out the galaxy help forum at https://help.galaxyproject.org/ to ask specific questions.

I would also personally advice that there are a number of interesting gotchas at times when running galaxy in a container due to it's breadth and complexity, especially with GIEs. If you are having issues, it may be beneficial to spin up a VM and try installing it there and going from there. This way it also better mimics the normal galaxy setup so the documentation is slightly more straightforward to apply.

Good luck!

[zhixuqiu]

OK,thanks your suggestion.

[bgruening]

The new 24.1 image contains a lot of changes and reflects the latest developments in Galaxy. I would like to close this PR, but please feel free to reopen and rebase against the latest version.

Please give it a try:

docker run -p 8080:80 -p 8021:21 -p 4002:4002 --privileged=true -e "GALAXY_DESTINATIONS_DEFAULT=slurm_cluster_docker"   -v /tmp/galaxy-data/:/export/ quay.io/bgruening/galaxy:24.1

... or any other combination. The readme has been updated. Please add any useful tip to it.

For a list of changes, see the Changelog.