Closed peterjc closed 7 years ago
Hi @peterjc, anything would be a great start. IMHO it would be great if we could use the galaxy-stable docker image at some point. TavisCI should be able to run a docker image with this nice trick https://github.com/lukecyca/travis-docker-example. We can save a lot of time setting up the Galaxy instance and at the same time have a reproducible instance.
The second improvement on my ToDo list is a Galaxy simple script that will convert a tool_dependencies.xml file into the bash script that is executed in the end from Galaxy. This could be used to install the tools or in a later step to generate Dockerfiles automatically.
That command line script to process a tool_dependencies.xml
idea has a lot of applications, including perhaps helping to automated turning Galaxy packages into other Linux packages?
Exactly ... just need time to study the code ... too bad @gregvonkuster has left the team.
I think that Tool Shed package installation should be done via API (or BioBlend) to maximize error reproducibility.
If you want to install packages, it might worth to have a look at Drone.io. For every build they create a docker container and btw their project is open source. I know that in the open source version one can specify its own docker image.
@nsoranzo at first we need to get our tools into one Tool Shed. The question is at which time to we want to test the tools. Should Travis-CI build a ToolShed, upload to all files to it and than start testing? Sounds ideal but maybe this like using a sledgehamer to crack a nut. The TestToolShed is also testing for us at some point.
at first we need to get our tools into one Tool Shed. The question is at which time to we want to test the tools. Should Travis-CI build a ToolShed, upload to all files to it and than start testing? Sounds ideal but maybe this like using a sledgehamer to crack a nut.
You're right, this may be overkill, otherwise when can use a dedicated ToolShed, but that would mean exposing the corresponding API key in the .travis.yml file.
You know that it is possible to encrypt environment variables in the .travis.yml ?
Didn't know, thanks for the hint!
I think the consensus of the devteam is that we would like to move toward lighter weight and more general processes. My vision is that you should be able to test tools without the need of a full tool shed (and once we eliminate a full tool shed as a testing dependency - I would like to eliminate the need for a full Galaxy server as well). I consider Peter's Travis-CI setup to be a good deal closer to where I want to take things than something that bootstraps a new tool shed (e.g. an automated version of the workflow Greg described at the GCC).
Do what you guys feel you must - prior to my assimilation I was never one to modify my plans based on the abstract future plans of the Borg - but I did want to let you know where my head was at on this issue.
How about we start with TravisCI script like https://github.com/peterjc/galaxy_blast and https://github.com/peterjc/pico_galaxy which:
apt-get install
or otherwise)Here by 'manually' I mean without using a (local) Galaxy Tool Shed, but instead installing binaries to $PATH
and adding tools to tool_conf.xml
directly.
Later we could install tool dependencies by reading the tool_dependencies.xml
files using a script as suggested by @bgruening. To avoid needing to automate building a dependency graph, in the first instance I would suggest .travis.yml
contains a series of manually selected dependencies but each simplified to something like process_tool_dependencies.py packages/package_numpy_1_7/tool_dependencies.xml
Once all the dependencies are handled directly, we ought to be able to walk the repository to generate tool_conf.xml
automatically?
I hope you guys consider working this into planemo. It takes care of some of these details (installing Galaxy, giving a spot to place custom installations, generation of tool_conf.xml
). It could provide a common place to do the more advanced scripting you described.
Example .travis.yml
for a very simple repository https://github.com/jmchilton/bwa-mem/commit/c7a5133f1d3574079c40f8e2c20b9202fb8f72af.
https://github.com/galaxyproject/planemo/issues/16 https://github.com/galaxyproject/planemo/issues/7 https://github.com/galaxyproject/planemo/issues/19
Might planemo be a good home for the proposed script to execute a tool_dependencies.xml
file outside of Galaxy?
@peterjc yes!
Yes, planemo should be the home for all kind of such hacks. I also have some script to convert argparse to tools.
https://github.com/galaxyproject/planemo/issues/15 is also relevant. That coupled with the docker-in-travis described above and we'd be almost there.
Additionally, if we're using planemo for that...we should be deploying to the toolshed when PRs are merged into master. :D
Okay y'all, I'm tired of seeing this issue, so....I set up jenkins testing in the meantime to test ALL of the tools. Yes it's overkill, but it's something, and it's working now until someone sets up Travis testing.
tool_dependencies.xml
globally. It'd be nice for planemo to implement that eventually, but this would provider a short term solution.Currently, two things (mostly) pass:
The rest I was too lazy to dig into tool_dependencies and actually install the backend packages, someone may want to do that....
Jenkins testing is now live and working. My apologies for all of you who got emails because the bot auto-commented on all open issues requesting an admin to verify a patch. I've removed or altered those comments to instruct jenkins to test where appropriate.
So, on to the neat stuff:
I'm going to see if I can build a jenkins plugin for galaxy tool testing. If I can, I'm willing to open this up as a free service to anyone above a threshold of some # of tools/packages. If that continues to work well, I'll further open the jenkins instance up to anyone who wants to use it for testing + deploying Galaxy tools.
If anyone cares about travis, please feel free to work on that and I'll quit with Jenkins.
As I see it, Pros of Jenkins vs Travis:
Cons of Jenkins vs Travis:
Neutral(ish) Items:
@jmchilton @erasche @bgruening @nsoranzo @peterjc Thanks for making this happen :+1:
Reading this again, shows that we have gone a long way! Thanks a bunch to the entire Galaxy community to make all this happen what was required to get this issue done!
Suggested by @bgruening, it would be great to have the IUC tools tested automatically using TravisCI.
Note that my setup described here http://blastedbio.blogspot.jp/2013/09/using-travis-ci-for-testing-galaxy-tools.html as used on https://github.com/peterjc/galaxy_blast and https://github.com/peterjc/pico_galaxy does not use the Tool Shed packages - it "manually" installs the dependencies from source or using "apt-get install".
What would be ideal for the IUC tools (and my own tools in the long term) would be to have the TravisCI script setup a local Galaxy and a local Tool Shed, and use this to test our IUC packages as well.