GCC 2016 Talk: The Intergalactic Utilities Commission - driving Galaxy tool development

bgruening commented 8 years ago

Hi IUC members,

we will have the chance to present the IUC at the upcoming GCC 2016 conference. This is the accepted abstract:

Galaxy provides abstractions to make it easy to integrate tools, so virtually any tool that can be run from 
the command line can be integrated into Galaxy. The ability to seamlessly integrate tools into Galaxy 
spawned a large community of Galaxy tool developers, with the Galaxy Tool Shed as a distrubtion 
platform for installation into any Galaxy instance. This proliferation of tools resulted in the need for an 
oversight committee to set standards, define best practices, and vet tools for the Galaxy community. In 
2012, the Intergalactic Utilities Commission (IUC) was founded as an organized body to provide these 
services, and has developed best-practice guidelines for tool development. These standards are a 
continual work-in-progress as new technologies are introduced into the Galaxy environment.

We will highlight IUC achievements over the past year, including enhanced reproducible installations via 
Starforge and cargo-port, new dependency resolution systems like Conda, and various enhancements to 
Galaxy tool syntax that enable more powerful and user-friendly tools. We’ll introduce new processes that 
have enhanced Galaxy tool development, testing and maintenance using Planemo and Conda, with 
details about how these applications can be used as complementary components to Galaxy and the 
Galaxy Tool Shed.

Important goals of the IUC are to continue to grow not only the community, but also the committee itself so 
that we can provide the friendly oversight benefits to every Galaxy Tool developer that is interested. This 
past year the IUC has welcomed 3 new members and organised 3 Codefests. We welcome others that 
have an interest in joining this committee and work with us.

The slides can be found here and you all are invited to contribute: http://bit.ly/1tDuFXs I will work during the next week on them and plan to finish them on Monday the 27th.

Here are a few points I will mention please add more:

IUC members, over the last years
IUC is a good starting point to contribute to Galaxy and to learn the large Galaxy ecosystem, many IUC members started as tool devs and are now Core-Galaxy contributors
Standards, Documentation, Maintenance of tools
Adopt orphan tools
planemo, cargo-port, depot, community packages, but also fixing upstream bugs and involving upstream
Tool-Dev training
mass conversion & adoption of new features (citations, CDATA ...)
driving new tool features (argument="", ...)
IUC Hackathons
- Collections
- Metagenomics
- RAD-seq
- Conda packaging
Highlight big community efforts like mothur, qiime, stacks
many IUC tools are featuring now usegalaxy.org

I plan to get as many IUC members together at GCC and talk about the future direction, especially how we go forward with dependency handling. If we decide something I would like to announce it during the end of the talk, e.g.:

The future is now:
- conda packages (bioconda, conda-forge)
- docker containers (mulled)
- automatic testing and CI integration of all tools
- automatic ToolShed upload

bgruening commented 8 years ago

Know as we know how the stage looks like do we want to be all on stage for this talk?

hexylena commented 8 years ago

Could!

yhoogstrate commented 8 years ago

@bgruening maybe flatpak and snap/snappy are interesting too

bgruening commented 8 years ago

@yhoogstrate not really applicable for HPC yet, and Galaxy has no support for it. Afaik there will be a way to convert docker to flatpack as needed. And with mulled we can support all this: Conda -> Docker -> ...

bgruening commented 8 years ago

Please review the slides. I think they are ready, any comments welcome.