search

galaxyproject / tools-iuc

Tool Shed repositories maintained by the Intergalactic Utilities Commission
https://galaxyproject.org/iuc
MIT License
160 stars 421 forks source link

Beacon tool to ask for variants #1011

Open bgruening opened 7 years ago

bgruening commented 7 years ago

We need a tool for this: https://beacon-network.org

A simple tool that uses the API to ask question to beacon.

ping @nekrut @jgoecks @jxtx

jgoecks commented 7 years ago

@bgruening Agreed, this would be great to have. Do you have other ideas for engaging with Beacon and/or GA4GH?

bgruening commented 7 years ago

@jgoecks Beacon will allow more fine-grained queries based on permissions in the near future, this can be handled by an extension which we currently develop for Galaxy to add arbitrary key-values into a user-preference object. Means, that a user can specify beacon-key and get more information from this service.

Moreover, I think that we have a better Docker-tooling as GA4GH wich we could offer them, @jmchilton was working on a talk afaik. But this is all above my pay grade ... Let me know if you need any help or more ideas :)

jgoecks commented 7 years ago

@bgruening We've traditionally not put user credentials into the Galaxy database so that Galaxy does not become a single point of failure and/or require substantial additional security measures.

I was just thinking that someone should do a short comparison of Galaxy Docker tooling compared to Dockstore tooling and then propose how to (a) improve Galaxy tooling using Dockstore functionality and/or (b) how to incorporate Galaxy Docker tooling into Dockstore.

jxtx commented 7 years ago

@bgruening I don't see how this is above your pay grade at all. Why not present your tooling to the containers and workflows working group? I talked to @briandoconnor a bit about this at #biodata16 and my understanding is dockstore is mainly focused on being a registry which could happily use our tools and containers.

bgruening commented 7 years ago

@bgruening We've traditionally not put user credentials into the Galaxy database so that Galaxy does not become a single point of failure and/or require substantial additional security measures.

It's a plugin, we can store the credentials wherever we want. It does not need to be in the database, any ideas welcome.

I was just thinking that someone should do a short comparison of Galaxy Docker tooling compared to Dockstore tooling and then propose how to (a) improve Galaxy tooling using Dockstore functionality and/or (b) how to incorporate Galaxy Docker tooling into Dockstore.

I think @jmchilton was planning to do this and give a talk about Galaxy + Docker at the GA4GH. This all gets to political for me, I just wanted to create a coherent and efficient deployment strategy with as less maintenance than possible - as we do not have any maintenance I think we succeeded here :)

bgruening commented 7 years ago

@bgruening I do t see how this is above your pay grade at all. Why not present your tooling to the containers and workflows working group? I talked to @briandoconnor a bit about this at #biodata16 and my understanding is dockstore is mainly focused on being a registry which could happily use our tools and containers.

Happy to discuss this with everyone, it's just that I'm not confident enough to knock on all doors from all registry creators - there is also bioshadock, iPlant and all of them create registries.

(Not sure why registries are important - I'm happy with quay.io - but what really matters to me is how containers are created and if this process is reproducible)

I was in contact with iPlant and they wanted to use mulled (biocontainers now) based containers. Not sure what the status is.

@jmchilton what are your plans for this presentation?

jxtx commented 7 years ago

I would be very happy to see @iplantcollaborativeopensource using mulled/biocontainers. It's about as low maintenance and reproducible a solution as one can imagine, and the more we can all share our tool/dependency/container/reproducibility story the better.

(And so he pings @mwvaughn @edwins)

edwins commented 7 years ago

@jxtx A while ago we looked into integrating mulled/biocontainers, but there were challenges with quay.io, a dependency for mulled at the time. We'll look into mulled/biocontainers again to see if those challenges still exist.

bgruening commented 7 years ago

Hi @edwins!

Quay.io is not a dependency of mulled or biocontainers it was just the most convenient way for us to create automatic containers via an API. Dockerhub did not offered this feature (does it now?). For 1800 containers you don't want to create repositories by hand :)

But the concept is completely independend of any hub. We have some documentation to create containers for entire conda-channels, for local conda packages and so on:

https://github.com/galaxyproject/galaxy/blob/dev/doc/source/admin/mulled_containers.rst

bgruening commented 7 years ago

Here is a first prototype implementation: https://github.com/galaxyproject/tools-iuc/pull/1066

@galaxyproject/iuc this tool is more or less asking a webserver and do not give any reproducible results. Please let me know if we should store this tool somewhere else.