./vpipe --dryrun error - Githubissues

ibseq commented 3 years ago

Hi all, I run the above with own data but got : (vpipe_env) ibseq:~/testing/work$ ./vpipe --dryrun VPIPE_BASEDIR = /univ/ibseq/testing/V-pipe AssertionError in line 68 of /univ/ibseq/testing/V-pipe/rules/common.smk: ERROR: Line '13' does not contain at least two entries! File "/univ/ibseq/testing/V-pipe/vpipe.snake", line 11, in File "/univ/ibseq/testing/V-pipe/rules/common.smk", line 68, in

from https://cbg-ethz.github.io/V-pipe/tutorial/sars-cov2/ any advice? thanks ibseq

carolinasisco commented 2 years ago

Hello,

I see you made this question almost a year ago but I´m having a similar issue:

VPIPE_BASEDIR = /home/geninfo/msisco/V-pipe/workflow ModuleNotFoundError in line 21 of /home/geninfo/msisco/V-pipe/workflow/rules/common.smk: No module named 'snakemake.sourcecache' File "/home/geninfo/msisco/V-pipe/workflow/Snakefile", line 12, in File "/home/geninfo/msisco/V-pipe/workflow/rules/common.smk", line 21, in

How did you solved it?

Thanks

Carol

kpj commented 2 years ago

@carolinasisco: which version of Snakemake are you using? Upgrading to the latest one (v6.13.1) could already solve your issue.

carolinasisco commented 2 years ago

Hi, thank you for your reply. The snakemake version that was downloaded (following V-pipe installation manual) was 3.13.3. How can I upgrade it inside my V-pipe conda environment?

Also I´m running a pipeline for SARS-CoV-2 but using a different reference than the one stated on V-pipe, besides the config and metainfo files, is there another scrip that will need to be modified?

Again, thank you for your help

DrYak commented 2 years ago

Hi,

The snakemake version that was downloaded (following V-pipe installation manual) was 3.13.3

The wiki is saddly out-dated - it matches older version 1.x and 2.x of V-pipe - and is in the process of being deprecated. (We should add "Warning" boxes about that, but hadn't had time yet).

More up-to-date example of installation include:

SARS-CoV-2 tutorial, also old but mostly similar to the current latest master branch
the Usage section of the main README.md, which has a subsection about the installer
the short documentation of the quickinstall in the utils' README.md

How can I upgrade it inside my V-pipe conda environment?

Note:

if you're only using conda for V-pipe, I would suggest deleting your current install and using the quick install script again.
if you have a complex conda installation, I would suggest making a separate conda environment containing just the latest versions of mamba and snakemake.
otherwise here are the detailed instructions:

First make sure that all the relevant channels are enabled, you need conda-forge and bioconda, with a higher priority for the former:

# set the channel precedence (lowest to highest)
conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
# NOTE conda-forge *HAS TO* be higher than bioconda

install mamba as this is now the conda package manager prefered by recent snakemake:

conda install mamba

now try updating snakemake:

mamba update --no-pin snakemake

warning: the no-pin allows to install also newer version of python, which might be required by more recent versions of snakemake, but might accidentally break other older package that you might have installed too.

Also I´m running a pipeline for SARS-CoV-2 but using a different reference than the one stated on V-pipe, besides the config and metainfo files, is there another scrip that will need to be modified?

Also of interest, once you have installed it, check the README.md in the config about how to configure it, and the latest documentation of the configuration format in config/config.html file in your local download (saddly, doesn't work online yet).

Specifically for your use case, check the options that are listed in the sars-cov2.yaml virus configuration as an inspiration in what you might want to adapt to your case.

carolinasisco commented 2 years ago

Hi,

I deleted the current install and re-installed it using the quick install script. However when running dry--run I get this error: FileNotFoundError in line 279 of /home/geninfo/msisco/V-pipe/workflow/rules/comm [Errno 2] No such file or directory: 'config/samples.tsv' File "/home/geninfo/msisco/V-pipe/workflow/Snakefile", line 12, in File "/home/geninfo/msisco/V-pipe/workflow/rules/common.smk", line 279, in

Note that the samples file has been created and is located inside the workflow directory, it just seem that is not able to create the samples.tsv by checking the samples directory.

DrYak commented 2 years ago

It's subtle difference dur to the most recent recommanded behaviours for modern snakemake workflows (see here: https://snakemake.readthedocs.io/en/stable/snakefiles/deployment.html) - current recommendations are to put the TSV files in the config/ directory.

If you're running a recent version of V-pipe (2.99.1 or master), out-of-the box V-pipe will follow these recommendations and look for the tsv file in config/samples.tsv. (But will not find this directory in your case).

If you prefere the older location (samples.tsv, without a directory), you can set it in the configuration file, in the section "input" -> property "_samplesfile".

See the section about the changes sine V-pipe 1.0/2.0 in the config's README.md.

carolinasisco commented 2 years ago

Hi, I keep getting this error:

VPIPE_BASEDIR = /home/geninfo/msisco/V-pipe/workflow No virus base configuration, using defaults AttributeError in line 230 of /home/geninfo/msisco/V-pipe/workflow/rules/common.smk: 'NoneType' object has no attribute 'setdefault' File "/home/geninfo/msisco/V-pipe/workflow/Snakefile", line 12, in File "/home/geninfo/msisco/V-pipe/workflow/rules/common.smk", line 259, in File "/home/geninfo/msisco/V-pipe/workflow/rules/common.smk", line 230, in process_config File "/home/geninfo/msisco/miniconda3/lib/python3.9/site-packages/jsonschema/validators.py", line 253, in validate File "/home/geninfo/msisco/miniconda3/lib/python3.9/site-packages/jsonschema/validators.py", line 229, in iter_errors File "/home/geninfo/msisco/miniconda3/lib/python3.9/site-packages/jsonschema/_validators.py", line 332, in properties File "/home/geninfo/msisco/miniconda3/lib/python3.9/site-packages/jsonschema/validators.py", line 245, in descend File "/home/geninfo/msisco/miniconda3/lib/python3.9/site-packages/jsonschema/validators.py", line 229, in iter_errors

Any advice?

thanks

DrYak commented 2 years ago

Hi, we have updated the config's readme section about the samples TSV to make more clear some problem that can arise if the read-length isn't correctly set.

DrYak commented 2 years ago

@carolinasisco :

AttributeError in line 230 of /home/geninfo/msisco/V-pipe/workflow/rules/common.smk: 'NoneType' object has no attribute 'setdefault' …. File "/home/geninfo/msisco/miniconda3/lib/python3.9/site-packages/jsonschema/validators.py", line 253, in validate

It's a problem when trying to validate your configuration. Something is broken in your config.yaml file (but saddly, the way the validation works in Snakemake, it doesn't give very useful messages).

Usually, you'll get 'NoneType' when you have an empty entry.

Something like:

general:
  virus_base_config:
    # empty no value here
output:
  snv: true
  …

in this example, section general property _virus_baseconfig will be a NoneType as there is no value assigned to it. (as opposed to output which is a section - it contains multiple properties - or svn - which is a Boolean: true/false).

carolinasisco commented 2 years ago

Usually, you'll get 'NoneType' when you have an empty entry

Is there a way to fix it? I really need to use this program

Thank you for all your help

DrYak commented 2 years ago

I really need to use this program

I know we're on different timezones, but you could quick drop on the chat at: https://gitter.im/V-pipe/community And we could quickly organise, e.g.: a Zoom meeting so you can show me what's exactly on your side.

cbg-ethz / V-pipe

./vpipe --dryrun error #80