Unable to run the workflow

[labrat9]

Hello, I tried installing mgefinder using both conda and pip but for either method, I encountered the following error:

mgefinder workflow denovo workdir/

Usage: mgefinder workflow [OPTIONS] WORKDIR Try 'mgefinder workflow --help' for help. Error: Invalid value for 'WORKDIR': Path 'denovo' does not exist.

And when I removed "denovo", I got another error:

mgefinder workflow workdir/

PARAMETERS

command: workflow
workdir: workdir/
snakefile: /home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/mgefinder/workflow/Snakefile configfile: /home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/mgefinder/workflow/config.yml
cores: 1
memory: 16000
unlock: False
rerun_incomplete: False
keep_going: False ###################
Traceback (most recent call last):
File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/bin/mgefinder", line 11, in sys.exit(cli())
File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/click/core.py", line 829, in call return self.main(args, kwargs)
File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx)
File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params)
File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(args, **kwargs)
File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/mgefinder/main.py", line 46, in workflow _workflow(workdir, snakefile, configfile, cores, memory, unlock, rerun_incomplete, keep_going) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.7/site-packages/mgefinder/workflow.py", line 7, in _workflow force_incomplete=rerun_incomplete, keepgoing=keep_going) TypeError: snakemake() got an unexpected keyword argument 'configfile'

Could you please advise?

Many thanks.

[durrantmm]

It looks like your version of MGEfinder is out of date. You can check this by first activating the conda environment with

conda activate mgefinder

And then open the python interpreter with

python

and then checking the version:

from mgefinder import __version__
print(__version__)

It should say 1.0.6. If it says something else, try completely uninstalling the environment with

conda env remove -n mgefinder --yes

And then re-downloading and installing according to these instructions.

[labrat9]

I did the re-downloading and installing according to the instruction, the version is confirmed to be 1.0.6, but still experienced error as seen below. Greatly appreciate your help.

Provided cores: 1 Rules claiming more threads will be scaled down. Job counts: count jobs 1 all 1 clusterseq 1 genotype 1 makefasta 1 summarize 5

rule clusterseq: input: workdir/01.mgefinder/I60/I60.all_inferseq_database.txt output: workdir/03.results/I60/01.clusterseq.I60.tsv jobid: 3 wildcards: genome=I60

Traceback (most recent call last): File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/bin/mgefinder", line 8, in sys.exit(cli()) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(args, kwargs) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(args, kwargs) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/main.py", line 308, in clusterseq _clusterseq(inferseqfiles, minimum_size, maximum_size, threads, memory, output_file) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/clusterseq.py", line 20, in _clusterseq inferseq = combine_inferseq_files(inferseqfiles, minimum_size, maximum_size) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/clusterseq.py", line 49, in combine_inferseq_files click.echo('Loading file {num1}/{num2}: {f}'.format(num1=1, num2=len(inferseq_files), f=inferseq_files[0])) IndexError: list index out of range Error in job clusterseq while creating output file workdir/03.results/I60/01.clusterseq.I60.tsv. RuleException: CalledProcessError in line 259 of /home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/workflow/denovo.original.Snakefile: Command ' mgefinder clusterseq -minsize 70 -maxsize 200000 --threads 1 --memory 16000 workdir/01.mgefinder/I60/I60.all_inferseq_database.txt -o workdir/03.results/I60/01.clusterseq.I60.tsv ' returned non-zero exit status 1. File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/workflow/denovo.original.Snakefile", line 259, in __rule_clusterseq File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/concurrent/futures/thread.py", line 56, in run Will exit after finishing currently running jobs. Exiting because a job execution failed. Look above for error message Traceback (most recent call last): File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/bin/mgefinder", line 8, in sys.exit(cli()) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(args, kwargs) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(args, kwargs) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/main.py", line 78, in denovo _workflow(workdir, snakefile, configfile, cores, memory, unlock, rerun_incomplete, keep_going) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/workflow.py", line 25, in _workflow shell(cmd) File "/home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/snakemake/shell.py", line 88, in new raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'snakemake -s /home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/workflow/denovo.original.Snakefile --config wd=workdir/ memory=16000 --cores 1 --configfile /home/styphi/sf_D/Test_dir/miniconda2/envs/mgefinder/lib/python3.6/site-packages/mgefinder/workflow/denovo.original.config.yml ' returned non-zero exit status 1.

[durrantmm]

Ok, this is an error that usually arises because it didn't find any insertions. How many samples are you using? Please check the files with the inferseq suffix to see if they found any insertions.

[labrat9]

I ran three different samples. The pipeline actually stopped after the first sample and did not analyse the other two.

[durrantmm]

Can you show me the structure of your working directory and subdirectories?

[labrat9]

This is my structure:

workdir/ ├── 00.assembly │   ├── I60.fna │   ├── I87.fna │   └── I90.fna ├── 00.bam │   ├── I60.ASM23692v1_genomic.bam │   ├── I60.ASM23692v1_genomic.bam.bai │   ├── I87.ASM23692v1_genomic.bam │   ├── I87.ASM23692v1_genomic.bam.bai │   ├── I90.ASM23692v1_genomic.bam │   └── I90.ASM23692v1_genomic.bam.bai └── 00.genome └── ASM23692v1_genomic.fna

3 directories, 10 files

[durrantmm]

Ok, that looks good. And could you show me how many lines are in each of your inferseq files?

[labrat9]

Sorry, I do not have any inferseq file. In the workdir, I have a new directory called mgefinder/GCA_000236925/I60/log Inside the directory, I have two files called I60.GCA_000236925.find.log and I60.GCA_000236925.find.log.err

[labrat9]

I tried redoing the sampling and still hit the same error...

[durrantmm]

Can you tell me the output of the file I60.GCA_000236925.find.log.err?

[labrat9]

I have added .txt file extension to enable it being uploaded.

I60.GCA_000236925.find.log.err.txt

[durrantmm]

Sorry, still not sure what's going wrong. If you uploaded the whole working directory in a zipped up directory I could better debug the problem.