Fix numpy error calling multiQC

[TomKellyGenetics]

Add conda environment for multiqc to resolve clash with base numpy.

[seb-mueller]

Thanks for the suggestions: The checks however failed:

WorkflowError:

Conda env file does not exist: /home/travis/build/Hoohm/dropSeqPipe/rules/../envs/multiqc.yaml

The command "snakemake --use-conda --directory .test -p" exited with 1.

It seems you you have added a conda environment conda: '../envs/multiqc.yaml' to some rules without creating a multiqc.yaml. This is usually done through the multiqc wrapper, '0.36.0/bio/multiqc'. So I'd suggest to add the ../envs/multiqc.yaml file, or fix the wrapper somehow.

Update: You seem to have the env in the wrong name (smk instead of yaml). But in general I'm not sure if it is a good idea to have both, a wrapper and a conda line in the rules. Do you think there is something wrong with the wrapper in general? I'd have to check. What's the base numpy clash exactly you are referring to?

[TomKellyGenetics]

Sorry, I'm not sure what the wrapper is meant to be doing but it doesn't seem to be working. Adding a yaml file to my directory for dropSeqPipe-0.4 seemed to work. Sorry I haven't tested the development version. It needs to run multiqc an env as with 0.36.0/bio/multiqc it still called numpy from outside conda (python2 in my case from my PATH).

[seb-mueller]

@Hoohm , any take on this one yet? I haven't had an error yet for this, @TomKellyGenetics , how to you call snakemake? Do you use --use-conda. It might be due to a broken wrapper, which I'd prefer to fix rather than replace with a custom environment. The relevant wrapper links are below: https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/multiqc.html https://github.com/snakemake/snakemake-wrappers/blob/master/bio/multiqc/environment.yaml

Have you checked if the wrapper failed in isolation as well? What is the actual error you get?

[Hoohm]

So the difference between the wrapper and the local env file is that the wrapper basically provides the env file for you.

For some reason the multiqc wrapper was lagging behind for a while in the past but today it's on 1.7 which is pretty close to 1.8.

The best scenario here would be to just update the wrapper version to "0.42.0/bio/multiqc".

Can you try that @TomKellyGenetics

[TomKellyGenetics]

I'm calling it with:

snakemake --use-conda --directory .

For some reason the multiqc environment was created but called dependencies from outside the environment. This fixes dropSeqPipe version 0.4 on my system (updating the wrapper to match the master branch didn't resolve it). I am testing whether the development version runs from the wrapper without a envs/multiqc.yamlfile. I will close this if it is not needed for the development version.

[TomKellyGenetics]

I've tested the develop branch (c50c484da) on the same server where I had issues running multiqc. The following wrapper seems to work in the current version.

wrapper:
        '0.36.0/bio/multiqc'

Therefore I will close this issue.

If anyone else runs into trouble running version 0.4 (2294701), the latest "release", then updating the wrapper does not work. You need to download an updated version on GitHub or add an environment file instead:

Add to MultiQC calls in rules/fastqc.smk, rules/map.smk, and rules/filter.smk

conda: '../envs/multiqc.yaml'

envs/multiqc.yaml

 channels:
 - conda-forge
 - bioconda
dependencies:
 - spectra
 - numpy
 - scipy
 - git
 - multiqc
 - perl
 - h5py
 - click

[seb-mueller]

Speaking of version 0.4, version 0.5 is already out but only as a tag rather than a "release". @Hoohm , could you make it as a release (I can try myself, but haven't done this yet, so not sure how)

https://github.com/Hoohm/dropSeqPipe/releases

[Hoohm]

Yes, but I want to have a real changelog for this :P

[TomKellyGenetics]

That would be great, I think a lot of the issues reported to the github repo are from version 0.4 (which is still directed to install). I was able reproduce many of the issues there running the older version.