Use galaxyproject.galaxy role to install and configure Galaxy.

[jmchilton]

This would bring docker-galaxy-stable closer inline with planemo-machine.

[mvdbeek]

You could also use galaxy-os for the packages and users, and ansible-galaxy-extras now also manages the server state, so I think almost everything in the Dockerfile can go (but that depends how radical you want to be ...). If you you agree i could give it a crack. Ping @bgruening

[jmchilton]

I want to be very radical :smirk: - I want the difference between planemo-machine, galaxy-ie-machine, and docker-galaxy-stable to be default values in an ansible variables YAML file.

[mvdbeek]

Then take a look at https://github.com/ARTbio/ansible-artimed You could look at the group_vars directory, for example aws and travis_bioblend build using the same playbook, and I got this to build docker images from it https://github.com/ARTbio/artimed-docker

[bgruening]

I'm fine with this. I mean this is what we are doing since one year, right? Moving stuff out of this project here and putting it in some playbooks. I kind of dislike how scattered this is currently. We need to update various roles and playbooks to get one feature implemented. Also no playbook is under testing currently but this will hopefully change when 16.01 is released. We should build various containers with the travis build-matrix to test this.

[mvdbeek]

@bgruening https://travis-ci.org/ARTbio/ansible-artimed is building and testing since yesterday (and with it the various playbooks we use), and the docker container of this playbook are building on travis (but no testing yet). I'll see how far I can get in making docker-galaxy!

[jmchilton]

@mvdbeek Exactly, using group_vars is exactly what I do in planemo-machine. Your stuff looks awesome - a lot like planemo-machine in some ways - except I am using CloudMan's postgres stuff instead of @natefoo's. We should rationalize the postgres stuff across all four or five projects also :smile:, I guess I should use Nate's stuff like you do.

@bgruening Hopefully as the development moves into the repositories adding features will entirely happen there - for instance I want to add this cool script Kyle against -extras that syncs a library with a directory. It would be cool if galaxy-docker-stable got that functionality with flick of an environment variable - ideally you wouldn't have to modify a thing.

You are right of course about testing and I think we are moving toward doing a better job. I'm going to try to enable some basic testing on planemo machine this iteration.

[mvdbeek]

I kind of dislike how scattered this is currently. We need to update various roles and playbooks to get one feature implemented.

Also we have a lot of slightly different variable names, and then the documentation is somewhere between non-existent and all over the place ... . I've started writing the docs for ansible-artimed, but I realised that if we could somehow parse all the # comments in front of variables we would be half-way there ...

@bgruening Hopefully as the development moves into the repositories adding features will entirely happen there - for instance I want to add this cool script Kyle against -extras that syncs a library with a directory. It would be cool if galaxy-docker-stable got that functionality with flick of an environment variable - ideally you wouldn't have to modify a thing.

Exactly, and in turn we benefit from state-of-the art IEs! ;) Plus in the future people could submit their flavours as a PR (in terms of preloaded tools and workflows), and get pre-built docker images and cloud machines or VMs or whatever.

[bgruening]

Exactly, and in turn we benefit from state-of-the art IEs! ;) Plus in the future people could submit their flavours as a PR (in terms of preloaded tools and workflows), and get pre-built docker images and cloud machines or VMs or whatever.

For the latter we already have a UI https://github.com/galaxyFlavorsGenerators/galaxyFlavorGenerator and some plans to integrate this in a community site to create Flavors. Let me know if you want to work on this :)

[mvdbeek]

I saw this, it's amazing, though I really don't know any JS. Would it be possible for the flavor generator to open a PR with the generated tool-list against the playbook repo, and then [insert any of travis, jenkins, ...] starts building and pushing the result to the docker hub [ or cloudlaunch ?! ] for the user to try. If you use an incremental build strategy this shouldn't take long. I'd be happy to see if we can auto-publish PRs to the docker hub.

[bgruening]

The backend it's not yet clear. But it will trigger a build of images and make them accessible. We are not clear if we can use Docker hub for it, or if we want to provide a Docker hub. Currently you can download the Dockerfile and compile yourself ;)

That said I don't think we need the github detour, we can build the image and push it somewhere.

[mvdbeek]

That said I don't think we need the github detour, we can build the image and push it somewhere.

Well, it could be optional, but let's say I (finally!) publish my paper, then it would be nice if I can have the set of tools and workflows that I used in a public, well-maintained repo, and I can be (kind-of) sure that people can find and use it with a recent version of galaxy. One way would be a PR with my flavour. If you look at homebrew's packages I think a model based on PRs could be successfull (especially if the PR is an option to click in the flavourGenerator).

We are not clear if we can use Docker hub for it, or if we want to provide a Docker hub.

You mean it could be too many/ too large images? If in doubt I'd try and see if they care about science :laughing: . Maybe they would even be very happy about it, who knows?

But thinking about this I also realize the importance of keeping the tiny pieces of community-crafted tool-lists instead of the GBs of docker images.

[bgruening]

@mvdbeek not sure if I understand the differences. Do you mean something like this: https://github.com/bgruening/docker-galaxy-ngs-preprocessing or the one from Greg https://github.com/gregvonkuster/docker-galaxy-csg

[mvdbeek]

Yes, but instead of a Dockerfile this would be a tool-list, a list of settings, perhaps a nice welcome.html, maybe pre-set for external PBS/Slurm, maybe with postgresql split out to the host or a HA-postgresql somewhere outside, with/without IEs, and (in addition to docker) for VMs or Cloud Machines or Bare Metal, and it would evolve together with the rest of the galaxy framework without a need for the original author to update anything. Then again, just because it's technically possible doesn't mean we should do it ;).

First step towards this direction would be to build as much as possible in ansible, because then we have a common basis (think "reference implementation") independent of the actual deployment technology, and then we see what makes sense in terms of user-experience.

[mvdbeek]

@jmchilton

for instance I want to add this cool script Kyle against -extras that syncs a library with a directory.

That's something i am looking for, you got some details?