Issues with multiple Qiime wrappers related to biom-format 2.1.5 lastest build (py27_3)

[arbernard]

Hi everyone,

I have an issue related to both the Qiime wrappers on the tools-iuc repo and to the Qiime Bioconda recipe (or rather one of its dependencies) so I'm not sure if this is the appropriate place to discuss it.

So, here is my issue:

Yesterday, I wanted to do a PR to fix some Datatype class not found errors in a few Qiime wrappers but even if the planemo tests on my local machine were running fine I stumbled upon issues during the travis build on my fork.

I checked the Travis log and the build was failing because it was detecting mismatches between test result files and the expected result files specified in the wrappers.

Nothing strange here... except that it was reporting mismatches for tools that I hadn't modified at all on my branch :/

Took me a while to troobleshoot the origin of the issue but it appears that the problem lies in the biom-format 2.1.5 bioconda recipe that is a prerequisite of the Qiime 1.9.1 recipe (which are both automatically installed if conda is activated on a given Galaxy instance that uses the Qiime IUC wrappers).

Indeed, if the tests were Ok on my local machine it was because my conda environment for Qiime 1.9.1 was using the py27_1 build of the biom-format 2.1.5 recipe while, in the meantime, they are failing with travis (or on new Galaxy instance I used for testing) because it is using the latest py27_3 build of the biom-format 2.1.5 recipe.

With this recent update of the biom-format recipe, every Qiime tools that used to generate biom1 (json) output files are now creating biom2 (binary) output files.

I haven't checked if this format change is causing issues in workflows that chains several of the affected tools yet but it obviously makes planemo tests and travis builds fail.

So, I don't know what is the best solution here.

Is it possible to request a specific build of a given prerequisite in a recipe ? (to revert to the previous situation where Qiime output files were not affected/modified)

Or should we try to deal with the changes introduced in the latest build of the biom-format 2.1.5 recipe ?

As discussed with @shiltemann yesterday on Gitter, adapting to the changes would at least mean that:

• The <tests> section of several wrappers must be updated along with the associated test files
• The <input> section of several wrappers must be updated to accept biom2 files where they were only accepting biom1 files until now (ie. when the Qiime tool itself can deal with biom2 files)
• An option that would allow the end user to choose between an output file in biom2 or biom1 format must be added (+ an auto-conversion command if he choose biom1)

There might be additional changes that I don't remember right now.

What do you think is the best solution here ?

ping @bebatut, @shiltemann, @bgruening

[nsoranzo]

@arbernard Can you open a WIP pull request, so that we can easily have a look at the Travis logs?

[arbernard]

@nsoranzo Ok, I'll create a WIP PR.