ERROR: Could not determine shovill version using '--version

[mabouelk]

Hi, I tried to install the nullarbor on python 3.7 but it says this conflict with the current python so I created an environment for it and activate it but upon check I got this error:

ERROR: Could not determine shovill version using '--version

I checked the shovill version separately and it is working "1.0.9"

[peflanag]

Hi @mabouelk I just did

conda remove --name nullarbor --all

Then when in the (base) shell I followed the steps to install it and the databases and it worked. However it only worked for one saureus set of samples I had. It wont work for TB samples as I get the error I posted here. So not sure if it was a fluke run!

https://github.com/tseemann/nullarbor/issues/258

But when I run nullarbor i run it like I stated in the post linked in this comment.

[mabouelk]

Thank you @peflanag and did you get a good output for aureus samples?

[peflanag]

Well I got an output but on the spp. ID it said it was unclassified so not sure if this is normal? I know that they are saureus and I was using the ref strain we would normally use! I am currently trying to run a TB batch and GC batch since these are the three organisms our lab deals with! I'm just not sure if the issues are cause I'm running ubuntu in parallel desktop on my mac and having nullarbor read the fastq from the mac and write to the linux desktop.

Ideally waiting on seeing if there is a fix for the shovill issue on macOS. I have installed it on a cluster but have yet to test it on the cluster cause thats a whole different story trying to write the .sh scripts to run it lol!

[tseemann]

Most clusters don't need you to write a shell script to submit. eg. sbatch on slurm You can do it directly from the command line and use options for RAM, CPUS etc eg. srun on slurm

to clarify - the --name nullarbor in your conda command is NOT the nullarbor conda package, that's the name of your conda environment. i usually use nullarbor_env for the env name so i don't get confused with the package name.

[mabouelk]

Hi @tseemann. Can you please help me with my issues: Installing it on mac os give me that error: Could not determine shovill version using '--version even I reinstall and updated shovill multiple times. installing it in an env, it asked me for centrifuge k something in cluster, I installed it successfully but after it runs, I got no output.

[tseemann]

I don't see any error reports at https://github.com/tseemann/shovill/issues

[tseemann]

Did you follow every step of https://github.com/tseemann/nullarbor#installation ?

[peflanag]

Sorry I never posted on shovill issues just commented here. When I run Nullarbor —check on macOS I get

ERROR: Could not determine shovill version using '--version

But I did follow every step and tried it a few times but got the same error. I’ll try again in the morning. When I do shovill —version it works but Nullarbor doesn’t seem to be able to access it of that makes sense. But I’ll check again in the morning cause for some reason I can’t log back into my work desktop

[mabouelk]

Hi @tseemann. You know I follow each step carefully and as I said I tried all the possible ways (my mac pro, cluster and env on conda). BTW, I am a big fan of you.

Right now I am reinstalling it again on virtual machine.

[tseemann]

# burn it all down
rm -fr ~/miniconda3
rm -fr ~/.conda
rm -fr ~/.condarc

# follow all these steps exactly
https://bioconda.github.io/user/install.html

# install latest fresh nullarbor
conda create -n nullarbor_env nullarbor
conda activate nullarbor_env
nullarbor.pl --check

[mabouelk]

Hi @tseemann, thank you for your help.

I followed your steps but I still get the same error (base) MOHAMEDs-MacBook-Pro:~ mohamedabouelkhair$ conda activate nullarbor_env (nullarbor_env) MOHAMEDs-MacBook-Pro:~ mohamedabouelkhair$ nullarbor.pl --check [18:35:35] Hello mohamedabouelkhair [18:35:35] This is nullarbor.pl 2.0.20191013 [18:35:35] Send complaints to Torsten Seemann [18:35:35] Found Perl module: Bio::SeqIO [18:35:35] Found Perl module: Cwd [18:35:35] Found Perl module: Sys::Hostname [18:35:35] Found Perl module: Time::Piece [18:35:35] Found Perl module: List::Util [18:35:35] Found Perl module: YAML::Tiny [18:35:35] Found Perl module: Moo [18:35:35] Found Perl module: Path::Tiny [18:35:35] Found Perl module: File::Copy [18:35:35] Found Perl module: File::Which [18:35:35] Found Perl module: File::Basename [18:35:35] Found Perl module: File::Spec [18:35:35] Found Perl module: File::Path [18:35:35] Found Perl module: Data::Dumper [18:35:35] Found Perl module: Term::ANSIColor [18:35:35] Found Perl module: SVG [18:35:35] Found Perl module: Text::CSV [18:35:35] Found Perl module: List::MoreUtils [18:35:36] Found Perl module: IO::File [18:35:36] Found 'head' => /usr/bin/head [18:35:36] Found 'cat' => /bin/cat [18:35:36] Found 'install' => /usr/bin/install [18:35:36] Found 'env' => /usr/bin/env [18:35:36] Found 'nl' => /usr/bin/nl [18:35:36] Found 'grep' => /usr/bin/grep [18:35:36] Found 'touch' => /usr/bin/touch [18:35:36] Found 'tr' => /usr/bin/tr [18:35:36] Found 'seqtk' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/seqtk [18:35:36] Found 'trimmomatic' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/trimmomatic [18:35:36] Found 'prokka' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/prokka [18:35:36] Found 'roary' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/roary [18:35:36] Found 'mlst' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/mlst [18:35:36] Found 'abricate' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/abricate [18:35:36] Found 'any2fasta' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/any2fasta [18:35:36] Found 'skesa' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/skesa [18:35:36] Found 'megahit' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/megahit [18:35:36] Found 'spades.py' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/spades.py [18:35:36] Found 'shovill' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/shovill [18:35:36] Found 'snippy' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/snippy [18:35:36] Found 'snp-dists' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/snp-dists [18:35:36] Found 'nw_order' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/nw_order [18:35:36] Found 'nw_display' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/nw_display [18:35:36] Found 'iqtree' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/iqtree [18:35:36] Found 'FastTree' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/FastTree [18:35:36] Found 'quicktree' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/quicktree [18:35:36] Found 'kraken' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/kraken [18:35:36] Found 'kraken-report' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/kraken-report [18:35:36] Found 'kraken2' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/kraken2 [18:35:36] Found 'centrifuge' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/centrifuge [18:35:36] Found 'centrifuge-kreport' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/centrifuge-kreport [18:35:36] Found 'fq' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/fq [18:35:36] Found 'fa' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/fa [18:35:36] Found 'roary2svg.pl' => /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/roary2svg.pl usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color=when] [--context[=num]] [--directories=action] [--label] [--line-buffered] [--null] [pattern] [file ...] [18:35:36] ERROR: Could not determine shovill version using '--version':

[mabouelk]

also I tried to run prokka to see if tbl2asn issue disappear or not? but I found it still exist

Running: tbl2asn -V b -a r10k -l paired-ends -M n -N 1 -y 'Annotated using prokka 1.14.6 from https://github.com/tseemann/prokka' -Z PROKKA_02252020\/PROKKA_02252020.err -i PROKKA_02252020\/PROKKA_02252020.fsa 2> /dev/null [18:57:04] Could not run command: tbl2asn -V b -a r10k -l paired-ends -M n -N 1 -y 'Annotated using prokka 1.14.6 from https://github.com/tseemann/prokka' -Z PROKKA_02252020\/PROKKA_02252020.err -i PROKKA_02252020\/PROKKA_02252020.fsa 2> /dev/null

[tseemann]

@mabouelk what does

echo $PATH | tr ":" "\n" | nl

say?

[mabouelk]

1 /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin 2 /Users/mohamedabouelkhair/opt/anaconda3/condabin 3 /usr/local/bin 4 /usr/bin 5 /bin 6 /usr/sbin 7 /sbin

[tseemann]

And this?

tbl2asn -  | head -n 3

I have:

tbl2asn 25.8   arguments:

If it is 25.7 something is wrong.

[mabouelk]

yes it is.

This copy of tbl2asn is more than a year old. Please download the current version.

tbl2asn 25.7 arguments:

[mabouelk]

however, I remember during installation, I saw tbl2asn-forever get installed

[tseemann]

what does ls -lsad $(which tbl2asn) say?

[mabouelk]

8 -rwxrwxr-x 2 mohamedabouelkhair staff 1208 Feb 3 08:59 /Users/mohamedabouelkhair/opt/anaconda3/envs/nullarbor_env/bin/tbl2asn

[peflanag]

I'm still getting the same issues on macOS and after running

tbl2asn - | head -n 3

I get

(nullarbor) peterflanagan@med176028 ~ % tbl2asn - | head -n 3

[NULL_Caption] This copy of tbl2asn is more than a year old. Please download the current version.

tbl2asn 25.7 arguments:

(nullarbor) peterflanagan@med176028 ~ %

But i cant seem to find a v25.8 for macOS . It only seems to be 25.7

[mabouelk]

I managed to install successfully on ubuntu using parellel thanks @peflanag and @tseemann . I run it now using 4 S. aureus fastq with the reference in genbank format. but I got this error [13:15:19] All nullarbor.pl 2.0.20191013 dependencies are installed. [13:15:19] You deserve a medal! [13:15:19] Loaded YAML config: /home/parallels/miniconda2/bin/../conf/nullarbor.conf [13:15:19] - nw_display = -S -s -w 1024 -l 'font-size:12;font-family:sans-serif;' -i 'opacity:0' -b 'opacity:0' -v 16 [13:15:19] - prefill = HASH(0x5565640bbe30) [13:15:19] - publish = /home/parallels/public_html/MDU [13:15:19] - trimmomatic = ILLUMINACLIP:/home/parallels/miniconda2/conf/trimmomatic.fa:1:30:11 LEADING:10 TRAILING:10 MINLEN:30 [13:15:19] Optimizing use of 2 cores for 4 isolates. [13:15:19] Will run concurrent 1 jobs with 2 threads each. [13:15:19] Preparing isolate rules and creating isolates.txt [13:15:19] Saving /home/parallels/run_Nullarbor_test/OUTDIR/nullarbor.log [13:15:19] Writing Makefile [13:15:19] Run the full pipeline with: [13:15:19] nice make all -j 1 -l 1 -C OUTDIR 2>&1 | tee -a OUTDIR/nullarbor.log [13:15:19] Done. (base) parallels@parallels-Parallels-Virtual-Platform:~/run_Nullarbor_test$ nice make all -j 1 -l 1 -C OUTDIR 2>&1 | tee -a OUTDIR/nullarbor.log make: Entering directory '/home/parallels/run_Nullarbor_test/OUTDIR' any2fasta -u /home/parallels/run_Nullarbor_test/sequence.gb > ref.fa touch --reference=/home/parallels/run_Nullarbor_test/sequence.gb ref.fa This is any2fasta 0.4.2 Opening '/home/parallels/run_Nullarbor_test/sequence.gb' Detected GENBANK format Read 95764 lines from '/home/parallels/run_Nullarbor_test/sequence.gb' Wrote 1 sequences from GENBANK file. Processed 1 files. Done. samtools faidx ref.fa ln -s -f '/home/parallels/run_Nullarbor_test/MRSA_CCH_101_R1.fastq.gz' 'Isolate1/R1.fq.gz' ln -s -f '/home/parallels/run_Nullarbor_test/MRSA_CCH_101_R2.fastq.gz' 'Isolate1/R2.fq.gz' fq --ref ref.fa Isolate1/R1.fq.gz Isolate1/R2.fq.gz > Isolate1/yield.tab [fq] running command: cat Isolate1/R1.fq.gz Isolate1/R2.fq.gz | seqtk fqchk -q0 - [fq] processed 11731766 reads. [fq] calculating depth, using size 2801000 (via --ref ref.fa) ln -s -f '/home/parallels/run_Nullarbor_test/MRSA_CCH_150_R1.fastq.gz' 'Isolate3/R1.fq.gz' ln -s -f '/home/parallels/run_Nullarbor_test/MRSA_CCH_150_R2.fastq.gz' 'Isolate3/R2.fq.gz' fq --ref ref.fa Isolate3/R1.fq.gz Isolate3/R2.fq.gz > Isolate3/yield.tab [fq] running command: cat Isolate3/R1.fq.gz Isolate3/R2.fq.gz | seqtk fqchk -q0 - [fq] processed 11949738 reads. [fq] calculating depth, using size 2801000 (via --ref ref.fa) ln -s -f '/home/parallels/run_Nullarbor_test/MRSA_CCH_166_R1.fastq.gz' 'Isolate4/R1.fq.gz' ln -s -f '/home/parallels/run_Nullarbor_test/MRSA_CCH_166_R2.fastq.gz' 'Isolate4/R2.fq.gz' fq --ref ref.fa Isolate4/R1.fq.gz Isolate4/R2.fq.gz > Isolate4/yield.tab [fq] running command: cat Isolate4/R1.fq.gz Isolate4/R2.fq.gz | seqtk fqchk -q0 - [fq] processed 12054812 reads. [fq] calculating depth, using size 2801000 (via --ref ref.fa) ln -s -f '/home/parallels/run_Nullarbor_test/MRSA_CCH_103_R1.fastq.gz' 'Isolate2/R1.fq.gz' ln -s -f '/home/parallels/run_Nullarbor_test/MRSA_CCH_103_R2.fastq.gz' 'Isolate2/R2.fq.gz' fq --ref ref.fa Isolate2/R1.fq.gz Isolate2/R2.fq.gz > Isolate2/yield.tab [fq] running command: cat Isolate2/R1.fq.gz Isolate2/R2.fq.gz | seqtk fqchk -q0 - [fq] processed 10970736 reads. [fq] calculating depth, using size 2801000 (via --ref ref.fa) read1="Isolate1/R1.fq.gz" read2="Isolate1/R2.fq.gz" outdir="Isolate1" cpus=2 opts="" /home/parallels/miniconda2/bin/../plugins/assembler/skesa.sh skesa --fastq Isolate1/R1.fq.gz,Isolate1/R2.fq.gz --cores 2 --vector_percent 1.0 --contigs_out Isolate1/contigs.fa

Total mates: 11731766 Paired reads: 5865883 Reads acquired in 59.373139s wall, 58.820000s user + 0.480000s system = 59.300000s CPU (99.9%) Adapters clip is disabled

Kmer len: 21 Raw kmers: 639261953 Memory needed (GB): 12.2738 Memory available (GB): 29.4694 1 cycle(s) will be performed /home/parallels/miniconda2/bin/../plugins/assembler/skesa.sh: line 11: 12550 Killed skesa --fastq "$read1,$read2" --cores "$cpus" --vector_percent 1.0 $opts --contigs_out "$outdir/contigs.fa" make: *** [Makefile:133: Isolate1/contigs.fa] Error 137 make: Leaving directory '/home/parallels/run_Nullarbor_test/OUTDIR'

[tseemann]

Can you please use make preview first? https://github.com/tseemann/nullarbor#quick-preview-mode I suspect your data files are bad, or skesa is not installed properly.

[mabouelk]

I did and the quality looks fine, all green check mark. How I can install skesa appropriately?

---

From: Torsten Seemann notifications@github.com Sent: Sunday, March 1, 2020 3:04:52 PM To: tseemann/nullarbor nullarbor@noreply.github.com Cc: Abouelkhair, Mohamed mabouelk@vols.utk.edu; Mention mention@noreply.github.com Subject: Re: [tseemann/nullarbor] ERROR: Could not determine shovill version using '--version (#254)

[External Email]

Can you please use make preview first? https://github.com/tseemann/nullarbor#quick-preview-mode I suspect your data files are bad, or skesa is not installed properly.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/tseemann/nullarbor/issues/254?email_source=notifications&email_token=AK7NPHQXSR5BG6TCSCCULUTRFK5WJA5CNFSM4KMWPJA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENNJJQQ#issuecomment-593138882, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK7NPHXXZPKAQ2ITL2BAV7LRFK5WJANCNFSM4KMWPJAQ.

[tseemann]

It's possible you ran out of RAM (memory) actually.

Does it run manually?

skesa --fastq MRSA_CCH_101_R1.fastq.gz,MRSA_CCH_101_R2.fastq.gz --vector_percent 1.0 --contigs_out CCH_101.contigs.fasta --cores 2

How much RAM and Cores do you have?

[mabouelk]

I got this:

Total mates: 11731766 Paired reads: 5865883Reads acquired in 55.972170s wall, 55.340000s user + 0.600000s system = 55.940000s CPU (99.9%)Adapters clip is disabledKmer len: 21Raw kmers: 639261953 Memory needed (GB): 12.2738 Memory available (GB): 29.4694 1 cycle(s) will be performed Killed

[tseemann]

Wow, you have ~12 million read pairs. For staph we only generate about 1.2 million, for 100x coverage you seem to have 1000x coverage. is that what the make preview report suggested in the DEPTH column?

you an try and downsample the data

seqtk sample MRSA_CCH_101_R1.fastq.gz > R1.fq.gz
seqtk sample MRSA_CCH_101_R2.fastq.gz > R2.fq.gz

and feed those new files to skesa or nullarbor.

OR

try giving skesa more memory --ram 64 if you have that much?

How much RAM and Cores do you have?

[mabouelk]

I have 32 cores and 64 GB RAM. I will try this and let you know.