MultiQC Conda Env Installs With Incompatible Python 3.12

felixm3 commented 1 year ago

Description of the bug

Trying to run the rnaseq pipeline on the test,conda profile.

Stops at multiqc step.

A web search shows that import imp which is needed by multiqc was deprecated in Python 3.12.

However, the conda env created by this nf-core workflow for running multiqc is created with Python 3.12.

How do I force this specific conda env creation step to use Python 3.11?

Command used and terminal output


nextflow run nf-core/rnaseq \
    -profile test,conda \
    --outdir ./231022_nf_test_01

Output


[3c/029348] process > NFCORE_RNASEQ:RNASEQ:MULTIQ... [100%] 1 of 1, failed: 1 ✘
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/rnaseq] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_RNASEQ:RNASEQ:MULTIQC (1)'

Caused by:
  Process `NFCORE_RNASEQ:RNASEQ:MULTIQC (1)` terminated with an error exit status (1)

Command executed:

  multiqc \
      -f \
       \
       \
      .

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_RNASEQ:RNASEQ:MULTIQC":
      multiqc: $( multiqc --version | sed -e "s/multiqc, version //g" )
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  Traceback (most recent call last):
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/bin/multiqc", line 6, in <module>
      from multiqc.__main__ import run_multiqc
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/lib/python3.12/site-packages/multiqc/__init__.py", line 16, in <module>
      from .multiqc import run
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/lib/python3.12/site-packages/multiqc/multiqc.py", line 30, in <module>
      from .plots import table
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/lib/python3.12/site-packages/multiqc/plots/table.py", line 9, in <module>
      from multiqc.plots import beeswarm, table_object
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/lib/python3.12/site-packages/multiqc/plots/beeswarm.py", line 8, in <module>
      from multiqc.plots import table_object
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/lib/python3.12/site-packages/multiqc/plots/table_object.py", line 9, in <module>
      from multiqc.utils import config, report
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/lib/python3.12/site-packages/multiqc/utils/report.py", line 18, in <module>
      import lzstring
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/lib/python3.12/site-packages/lzstring/__init__.py", line 11, in <module>
      from future import standard_library
    File "/home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/conda/env-26bbe61d6cbca5f71da2d7fcef2c52eb/lib/python3.12/site-packages/future/standard_library/__init__.py", line 65, in <module>
      import imp
  ModuleNotFoundError: No module named 'imp'

Work dir:
  /home/fmbuga/scATAC/data/Itokawa22HSC/HSC_RNAseq/work/3c/0293483830cd4d5c0e1ead8100cdab

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details

System information


Nextflow version 23.10
Hardware HPC
Executor local 
Container engine: conda
OS CentOS Linux
Version of nf-core/rnaseq 3.12.0

sinanugur commented 1 year ago

I got the same error when using-profile conda.

A quick temporary solution is activating the multiqc environment and downgrading the python version.

conda activate the_envrironment_containing_nextflow #this is visible on the error text
conda install python=3.11
conda deactivate

pinin4fjords commented 1 year ago

Fix included in https://github.com/nf-core/rnaseq/pull/1106

nf-core / rnaseq