Picard environment not created

[henriquevf]

When I run the command and it starts creating the required environments it gets stuck creating picard.yaml environment.

Creating conda environment ../envs/picard.yaml...
Downloading and installing remote packages.

I have tested the command separately and it keeps "solving environment" indefinitely.

conda env create -f picard.yaml
Collecting package metadata (repodata.json): done
Solving environment: |

I have tested other environments from grenepipe and they work. I also have tried to change some of the dependencies version, but no luck. My conda installation is relatively new and I don't have a lot of things installed on base, which usually causes the "solving environment" issue.

[henriquevf]

Using the latest version of the softwares to create the environment worked. Hope there won't be any conflicts when Picard is required. I have edited the yaml file.

name: picard_test

channels:
  - bioconda
  - conda-forge
  - r
dependencies:
  - picard ==2.26.2
  - snakemake-wrapper-utils ==0.2
  - python ==3.7.10
  - numpy ==1.20.3
  - pandas ==1.3.3
  - r-tidyverse ==1.3.1

[lczech]

Hi @henriquevf,

thanks for reporting this! The issue was introduced by fixing #9; I had noticed it a few days ago myself and already had a fix, but hadn't pushed it. My fix does not change the versions (as this might introduce some unintended problems), but instead removes some of the channels, which has worked in my tests. It takes a few minutes still on my machine, but works.

It would be great if you could download the latest version of grenepipe, and check if this fix works for you as well. And in any case, thank you very much for the thorough issue and description of your approaches!

Cheers and so long Lucas

PS: Unfortunately, conda is known to be slow in many cases. One solution is to use a relatively recent snakemake version, where you can use mamba instead via --conda-frontend: see here.

[lczech]

Hi @henriquevf, any news on this?

[lczech]

Hi @henriquevf,

I've worked on the conda environments of picard now (in the latest commits), and changed and fixed a few things. It seems to install now without issues.

Feel free to re-open this issue should you still have trouble when using the latest grenepipe (not released yet, so use the download or git clone button for now, or wait for grenepipe >= v0.7.0 to come out).

Lucas

[RvV1979]

Hi Lucas and all,

FYI I seem to have had this same issue when running the pipeline on the example directory using conda. I was using the versions below: Conda: 22.9.0 Python: 3.7.10 Snakemake: 6.0.5 Grenepipe: 0.10.0

Using these versions, creating the picard environment hangs forever...

Fortunately, when following your suggestion form an earlier answer above, specifying --conda-frontend mamba the issue was resolved.

[lczech]

Thanks for the report, @RvV1979! Fixing the picard environment somehow so that it works with normal conda is on my todo list, but given that mamba works fine, it's not very high priority. However, as it seems that this is an ongoing issue for many users (hence again thanks for reporting this!) I might increase the priority of this fix ;-)

[lczech]

Hi @henriquevf and @RvV1979,

I've revisited this issue again, see also #25, and hopefully found a solution now. The picard and qualimap environments are still slow with conda, but at least do not completely hang any more - at least in my tests. Unfortunately, this works-for-me approach is all that I can do - I'm currently running large scale tests on Ubuntu, CentOS, and MacOS, with conda, and with mamba (not all permutations, that would be too much, but the more important ones). Seems to be working well enough, but I'd also like to get your feedback on whether it works on your systems, in case this is still relevant for your.

If so, could you maybe try again using grenepipe at this commit?

Cheers and so long Lucas

[lczech]

Hi again, quick notice: I just published grenepipe v0.12.0, which (hopefully) fixes all these issues. The picard and the qualimap environments were the biggest culprits, and have been updated. All environments install fine on Ubuntu and an MacOS now, both with conda and with mamba ;-)