Issue with mark_duplicates. No module named 'distutils'

[htorrado]

Hello,

Thank you for your hard work on this package!

I've recently installed it and followed the instructions in the manual to run the test data. However, I encountered an error when executing the mark_duplicates rule (jobid: 27). The error seems to be related to loading the library distutils, although I can load it manually in Python without issues. Here’s the error message:

  File "/anaconda3/envs/findZX/lib/python3.9/site-packages/snakemake/__init__.py", line 22, in <module>
    from snakemake.workflow import Workflow
  File "/home/user/anaconda3/envs/findZX/lib/python3.9/site-packages/snakemake/workflow.py", line 33, in <module>
    from snakemake.shell import shell
  File "/home/user/anaconda3/envs/findZX/lib/python3.9/site-packages/snakemake/shell.py", line 19, in <module>
    from snakemake.deployment import singularity
  File "/home/user/anaconda3/envs/findZX/lib/python3.9/site-packages/snakemake/deployment/singularity.py", line 11, in <module>
    from distutils.version import LooseVersion
ModuleNotFoundError: No module named 'distutils'
Error in rule mark_duplicates:
    jobid: 27
    output: results/ref/AloPal_v1_subset/dedup/subset_SRR9655171__heterogametic.sorted.dedup.mismatch.unfiltered.bam, results/howler_monkey_test_dataset/qc/dedup/subset_SRR9655171__heterogametic.metrics.txt
    log: results/howler_monkey_test_dataset/logs/picard/dedup/subset_SRR9655171__heterogametic.log (check log file(s) for error message)
    conda-env: /home/user/Programs/findZX/.snakemake/conda/ad0ddccd975db9c41b21ea22863d03df

RuleException:
CalledProcessError in line 52 of /home/user/Programs/findZX/workflow/rules/mapping.smk:
Command 'source /home/user/anaconda3/envs/findZX/bin/activate '/home/user/Programs/findZX/.snakemake/conda/ad0ddccd975db9c41b21ea22863d03df'; set -euo pipefail;  python /home/user/Programs/findZX/.snakemake/scripts/tmp7hp8336f.wrapper.py' returned non-zero exit status 1.

I verified that the required software versions are installed correctly:

(findZX) ~/Programs/findZX$ python -V
Python 3.9.4

(findZX) ~/Programs/findZX$ mamba --version
mamba 0.15.3
conda 4.12.0

(findZX) ~/Programs/findZX$ snakemake --version
6.4.0

I also attempted to make some small modifications to the code, but they didn’t resolve the issue. I’ve tried running the code on a couple different computers with the same output, so the problem doesn’t seem to be specific to my local setup.

Could you please help me troubleshoot this issue?

Thank you in advance for your support!

Best regards, Héctor

[hsigeman]

Hi Héctor,

Thank you for your message and for notifying me know about this bug!

After cloning a new version of findZX I got exactly the same error as you. While I'm not entirely sure about the underlying cause, there seems to be a compatibility issue between the snakemake wrapper for picard and conda and/or snakemake.

From the snakemake issues page, there appears to have been a fix related to distutils last year, though it's unclear if or how it relates to this specific problem: https://github.com/snakemake/snakemake/issues?q=distutils

However, I was able to resolve the issue by creating a new Conda environment with more recent versions of snakemake and snakemake-wrapper-utils (I simply omitted the software versions to that the most recent ones were installed). The pipeline ran successfully in this setup. Could you please try the following steps and let me know if it works for you as well?

conda create -n findZX_test -c conda-forge -c bioconda snakemake-wrapper-utils snakemake mamba
conda activate findZX_test
snakemake -s workflow/findZX --configfile .test/config.yml --cores 1 -R all --use-conda -k --conda-frontend mamba

If this resolves the issue on your end, I’ll update the instructions in the README to reflect this fix.

Best regards, Hanna

[htorrado]

Hi Hanna,

Yes, that solved the problem and it ran successfully. Thank you very much!

Best regards, Héctor