vpipe broken? ModuleNotFoundError: No module named 'pandas'

schlaepferp commented 1 year ago

Issue

I tried to install vpipe. Running samples fails because it misses pandas. I guess it has to do with the latest release of minoconda?

OS:

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

Installation

mkdir vpipe_test_dir chmod a+rwx ./vpipe_test_dir cd ./vpipe_test_dir curl -O 'https://raw.githubusercontent.com/cbg-ethz/V-pipe/master/utils/quick_install.sh' chmod a+rwx ./quick_install.sh ./quick_install.sh -w work cd work scp /path/to/config.yaml . scp -r /path/to/samples . ./vpipe --jobs 1 --cores 2 --printshellcmds --dry-run ./vpipe --jobs 1 --cores 2 --printshellcmds

Content of config.yaml

general: virus_base_config: 'sars-cov-2'

input: samples_file: samples.tsv read_length: 150

output: trim_primers: false snv: true local: false global: false visualization: true diversity: false QA: false upload: false dehumanized_raw_reads: false

Place of failure

--> Fails at gunzip -c samples/FolderA/FolderB/raw_data/FolderA_FolderB_R2.fastq.gz > samples/FolderA/FolderB/raw_data/FolderA_FolderB_R2.fastq

Error message

Traceback (most recent call last): File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/init.py", line 770, in snakemake success = workflow.execute( File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/workflow.py", line 1088, in execute raise e File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/workflow.py", line 1084, in execute success = self.scheduler.schedule() File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/scheduler.py", line 541, in schedule self._finish_jobs() File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/scheduler.py", line 641, in _finish_jobs self.get_executor(job).handle_job_success(job) File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/executors/init.py", line 671, in handle_job_success super().handle_job_success(job) File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/executors/init.py", line 259, in handle_job_success job.postprocess( File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/jobs.py", line 1467, in postprocess job.postprocess(error=error, **kwargs) File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/jobs.py", line 1165, in postprocess self.dag.workflow.persistence.finished( File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/persistence.py", line 258, in finished params = self._params(job) File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/persistence.py", line 491, in _params return sorted( File "/home/user@server/vpipe_test_dir/miniconda3/lib/python3.10/site-packages/snakemake/persistence.py", line 483, in _serialize_param_pandas import pandas as pd ModuleNotFoundError: No module named 'pandas'

schlaepferp commented 1 year ago

I forgot to add:

Possible fix

Add the following lines at the end of the quick_install.sh source ./miniconda3/bin/activate conda install pandas conda deactivate

DrYak commented 1 year ago

Hi @schlaepferp!

This is an upstream bug in Snakemake: snakemake-minimal should NOT rely on pandas, but the logic for handling that had an implementation bug. I've helped Johannes fixing it and it is now available in snakemake version.

So the definitive fix is:

. ./miniconda3/bin/activate
mamba install "snakemake-minimal>=7.28.3"
conda deactivate

cbg-ethz / V-pipe