makedb.pl failed # error code 256

[Oakley-Lab]

Hi I have installed phyloFlash and tried to create database as per suggestion. But it failed on both linux and OS X with error code 256. Pls see below the output of terminal. Can you please help me to resolve the issue? Best, Jitendra ######################################################## sudo ./phyloFlash_makedb.pl --remote [18:34:11] Checking for required tools. [18:34:12] Using barrnapHGV found at "/bin/phyloFlash/barrnap-HGV/bin/barrnap_HGV". [18:34:12] Using grep found at "/bin/grep". [18:34:12] Using bowtiebuild found at "/bin/bowtie-build". [18:34:12] Using bbduk found at "/bin/bbduk.sh". [18:34:12] Using vsearch found at "/usr/local/bin/vsearch". [18:34:12] Using bbmap found at "/bin/bbmap.sh". [18:34:12] Using bbmask found at "/bin/bbmask.sh". [18:34:12] All required tools found.

This is phyloFlash_makedb.pl from phyloFlash.pl v3.3b1

[18:34:12] downloading latest univec from ncbi [18:34:12] Connecting to ftp.ncbi.nlm.nih.gov [18:34:12] Finding /pub/UniVec/UniVec [18:34:13] Found UniVec (1701925 bytes) [18:34:13] Found existing UniVec [18:34:13] Sizes match. Skipping download [18:34:13] downloading latest SSU RefNR from www.arb-silva.de [18:34:13] Connecting to ftp.arb-silva.de [18:34:14] Finding /current/Exports/*_SSURef_Nr99_tax_silva_trunc.fasta.gz [18:34:16] Found SILVA_132_SSURef_Nr99_tax_silva_trunc.fasta.gz (238828503 bytes) [18:34:17] Found existing SILVA_132_SSURef_Nr99_tax_silva_trunc.fasta.gz [18:34:17] Verifying MD5... [18:34:17] Verified. Skipping download [18:34:17] unpacking SILVA database [18:34:17] File ./132//SILVA_SSU.fasta exists, not overwriting [18:34:17] LSU-filtered file found, not overwriting [18:34:17] masking low entropy regions in SSU RefNR [18:34:17] running subcommand: /bin/bbmask.sh overwrite=t -Xmx10g threads=8 in=./132//SILVA_SSU.noLSU.fasta out=./132//SILVA_SSU.noLSU.masked.fasta minkr=4 maxkr=8 mr=t minlen=20 minke=4 maxke=8 fastawrap=0 2>tmp.bbmask_mask_repeats.log [18:34:17] FATAL: Tool execution failed!. Error was 'No such file or directory' and return code '256' Aborting. [18:34:17] Saving log to file phyloFlash_log_on_error

[kbseah]

Hello Jitendra, could you please attach the contents of the log file tmp.bbmask_mask_repeats.log?

[kbseah]

Hello Jitendra, I don't think you can include an attachment when replying by email. Your email address is also now exposed. You might want to edit your last reply in the Github web interface.

[Oakley-Lab]

tmp.bbmask_mask_repeats.log

[kbseah]

It looks like one of the dependencies, BBmap, is not working properly. It is not able to find the script calcmem.sh, which should be available when you install BBmap. How did you install the dependencies for phyloFlash?

[Oakley-Lab]

Thank you, re-added bbmap tools to the bin and trying makedb. Command is running, will update you after finishing.

Dependencies were added from the list below and added to the bin manually.

Best, Jitendra

phyloFlash relies on the following software:

Perl >= 5.13.2 EMIRGE and its dependencies BBmap Vsearch SPAdes Bedtools Mafft Barrnap (customized version is provided with phyloFlash)

[kbseah]

Good to know that it's working now. Do let us know if you encounter any more issues.

If you have time, you can give Conda a try for installing software. It handles things like dependencies and paths automatically.

[Oakley-Lab]

I appreciate assistance. I had tried before using conda but had issues with python version. Attached file.

Best regards, Jitendra

conda.install.problem.txt

[kbseah]

In this situation the fix is usually to create a new conda environment for the package. This allows you to isolate different software packages that have incompatible dependencies, because each lives in their own environment, which you can activate and deactivate as necessary.

You can try the following command

conda create --strict-channel-priority -c conda-forge -c bioconda -c defaults -n phyloflash phyloflash

For more information on why --strict-channel-priority is sometimes necessary see: http://hrgv.github.io/phyloFlash/install.html

[Oakley-Lab]

Execution of tool Vsearch is getting terminated. Please help me out.

Best, Jitendra

TEST.all.final.phyloFlash.notmatched.fa -dbmatched

   TEST.all.final.phyloFlash.dbhits.fa -db

   ./132/SILVA_SSU.noLSU.masked.trimmed.udb >TEST.all.vsearch.out

   2>&1

[11:46:08] FATAL: Tool execution failed!.

   Error was '' and return code '256'

vsearch.txt

[kbseah]

Hello Jitendra,

Could you please attach the file TEST.all.vsearch.out?

Off the top of my head, one possibility is that the database file ./132/SILVA_SSU.noLSU.masked.trimmed.udb may have different rights assigned (see https://github.com/HRGV/phyloFlash/issues/58), but if you're on a single-user laptop that usually won't be the case. The log file should say what went wrong.

Best, Brandon

[Oakley-Lab]

TEST.all.vsearch.txt

PFA, Could it be a less RAM issue? I am running on personal computer of 8Gb RAM.

Best, Jitendra

[kbseah]

Hm that could be the issue. Did you prepare the database files on the same computer? Because the initial database indexing should have used a similar amount of RAM.

Could you run vsearch --udbinfo ./132/SILVA_SSU.noLSU.masked.trimmed.udb and vsearch --udbstats ./132/SILVA_SSU.noLSU.masked.trimmed.udb?

The latter command should read the db index into memory and calculate statistics. If this doesn't complete properly then it could be insufficient memory. You can run Activity Monitor to see how much it is actually consuming.

We haven't really tested this software on Mac OS X, and recommend that you run it on Linux to be on the safe side.

[Oakley-Lab]

Thanks for the suggestion, will give a try on linux now. The suggested commands failed with Fatal error: Unable to read from UDB file or invalid UDB file. See below. However the database was created using same computer.

Best, Jitendra

######## (base) Jitendras-MacBook-Air:phyloFlash-pf3.3b1 jitendrakeshri$ vsearch --udbinfo ./132/SILVA_SSU.noLSU.masked.trimmed.udb vsearch v2.13.6_macos_x86_64, 8.0GB RAM, 4 cores https://github.com/torognes/vsearch

Fatal error: Unable to read from UDB file or invalid UDB file

(base) Jitendras-MacBook-Air:phyloFlash-pf3.3b1 jitendrakeshri$ vsearch --udbstats ./132/SILVA_SSU.noLSU.masked.trimmed.udb vsearch v2.13.6_macos_x86_64, 8.0GB RAM, 4 cores https://github.com/torognes/vsearch

Reading UDB file ./132/SILVA_SSU.noLSU.masked.trimmed.udb 0%

Fatal error: Unable to seek in UDB file or invalid UDB file

[kbseah]

What is the file size of the .udb file? It should be about 4.5 Gb. You'll need to have at least that much free RAM to run the pipeline. My guess is that the database indexing failed or got corrupted somehow, maybe because of memory limitation. Although your computer has 8 Gb total, some is already used by the OS and other tasks, so it could explain why it is insufficient.

Do give the Conda installation a try, it should be easier than installing the dependencies individually, but be sure to create a new environment for phyloFlash alone.

Hope that helps!

[Oakley-Lab]

Probably the problem was during indexing coz the file size is even bigger (5.03Gb). I will try installing with conda by creating an env for phyloflash.

Thanking you, Jitendra

[Oakley-Lab]

Thanks for helping, the pipeline is working after installation with conda.

Best, Jitendra

[whitnersyrena]

Hi there,

I'm trying to run phyloflash on a MacOS and having a similar issue while trying to create the initial database. It gets through to the bowtie build step and then gives the following error:

running subcommand: /opt/anaconda3/envs/pf/bin/bowtie-build ./138.1/SILVA_SSU.noLSU.masked.trimmed.NR96.fixed.fasta ./138.1/SILVA_SSU.noLSU.masked.trimmed.NR96.fixed.bt -q 2>tmp.bowtiebuild.log sh: line 1: 78886 Abort trap: 6 /opt/anaconda3/envs/pf/bin/bowtie-build ./138.1/SILVA_SSU.noLSU.masked.trimmed.NR96.fixed.fasta ./138.1/SILVA_SSU.noLSU.masked.trimmed.NR96.fixed.bt -q 2> tmp.bowtiebuild.log [14:57:38] FATAL: Tool execution failed!. Error was 'No such file or directory' and return code '34304' Check error log file tmp.bowtiebuild.log Aborting. [14:57:38] Saving log to file phyloFlash_log_on_error

Here is the information provided in the bowtiebuild.log file: dyld: Library not loaded: @rpath/libtbb.dylib Referenced from: /opt/anaconda3/envs/pf/bin/bowtie-build-s Reason: image not found

If anybody could please advise on how to solve this issue that would be GREATLY appreciated :)

Thanks, Syrena