Error 256 running command

[zhaoxvwahaha]

there is an error when I running the shovill, anybody could gimme a help? Read stats: max_len = 100 Read stats: avg_len = 100 Read stats: total_bp = 1200311600 Read stats: min_len = 100 Estimating genome size with 'kmc' Running: kmc -m32 -n256 -ci3 -k25 -t16 mypair1.gz kmc /tmp/9619922.1.st.q >> 20-kmc.log 2>&1 Error 256 running command

[tseemann]

It seems kmc is not working on your system.

Can you paste the contents of the 20-kmc.log file here please?

[zhaoxvwahaha]

I run several other samples, but all report the 256 error, like this: Checking for assembly errors in scaffolds.fasta Running: bwa index scaffolds.fasta >> 70-bwa.log 2>&1 Running: (bwa mem -v 3 -x intractg -t 16 scaffolds.fasta R1.fq.gz R2.fq.gz | samtools sort --threads 1 -m 100G --reference scaffolds.fasta -T /tmp/9838024.1.st.q/samtools.58882 -o shovill.bam) >> 70-bwa.log 2>&1 Error 256 running command

The command I used is: shovill --outdir abc --R1 pair1.fq.gz --R2 pair2.fq.gz --ram 200 --minlen 100 --asm scaffolds --trimopt LEADING:20 TRAILING:20

[tseemann]

This is now a different error? I notice samtools is asking for -m 100G of RAM ? Do you have 100 gigabytes of fre RAM on your computer?

What was the complete original command line you ran?

[zhaoxvwahaha]

Thank you for timely reply, All my tasks have the same error as I posted above. Because I submit the job by SGE to super computer cluster, perhaps the RAM is available when the source is vacant. The error 256 is related to the RAM problem? How to set the RAM of samtools, I don't think it need so large RAM, this value is automatically compute by your pipeline? The complete original command line is: software/anaconda2/bin/shovill --outdir abc --R1 pair1.fq.gz --R2 pair2.fq.gz --ram 200 --minlen 100 --asm scaffolds --trimopt LEADING:20 TRAILING:20

[tseemann]

Please do not use --ram 200 unless your computer nodes have 200 GB of RAM.

I am unable to help you with setting the correct parameters for your SGE queue.

[stephenturner]

I'm getting a similar error but at the spades step.

Running: spades.py --pe1-1 flash.notCombined_1.fastq.gz --pe1-2 flash.notCombined_2.fastq.gz --s2 flash.extendedFrags.fastq.gz --only-assembler --threads 16 --memory 100 -o . --tmp-dir /tmp -k 31,55,79,103,127  >> /dev/null 2>&1
Error 256 running command

When running that spades command manually, seems to work fine at k31, but fails at k55.

=== Stack Trace ===
/home/sdt5z/miniconda3/envs/mavium/share/spades-3.11.1-1/bin/spades(_ZN5utils16print_stacktraceEv+0x3a) [0x59799a]
/home/sdt5z/miniconda3/envs/mavium/share/spades-3.11.1-1/bin/spades() [0x7ff336]
/home/sdt5z/miniconda3/envs/mavium/share/spades-3.11.1-1/bin/../../../lib/libgomp.so.1(+0xfb05) [0x7f095bb64b05]
/lib64/libpthread.so.0(+0x7e25) [0x7f095b72ee25]
/lib64/libc.so.6(clone+0x6d) [0x7f095b45c34d]
Verification of expression 'f' failed in function 'void utils::KMerSortingSplitter<Seq>::DumpBuffers(const files_t&) [with Seq = RuntimeSeq<128ul>; fs::files_t = std::vector<std::basic_string<char> >]'. In file '/opt/conda/conda-bld/spades_1518344427800/work/SPAdes-3.11.1/src/common/utils/kmer_mph/kmer_splitters.hpp' on line 132. Message 'Cannot open temporary file to write'.
Verification of expression 'f' failed in function 'void utils::KMerSortingSplitter<Seq>::DumpBuffers(const files_t&) [with Seq = RuntimeSeq<128ul>; fs::files_t = std::vector<std::basic_string<char> >]'. In file '/opt/conda/conda-bld/spades_1518344427800/work/SPAdes-3.11.1/src/common/utils/kmer_mph/kmer_splitters.hpp' on line 132. Message 'Cannot open temporary file to write'.
spades: /opt/conda/conda-bld/spades_1518344427800/work/SPAdes-3.11.1/src/common/utils/kmer_mph/kmer_splitters.hpp:132: void utils::KMerSortingSplitter<Seq>::DumpBuffers(const files_t&) [with Seq = RuntimeSeq<128ul>; fs::files_t = std::vector<std::basic_string<char> >]: Assertion `f' failed.

== Error ==  system call for: "['/<path>/share/spades-3.11.1-1/bin/spades', '/<path>/K55/configs/config.info']" finished abnormally, err code: -6

[giriarteS]

I got the same error running kmc Hello xxx You ran: /usr/local/bin/shovill --outdir shovel --R1 Clav04_S1_L001_R1_001.fastq.gz --R2 Clav04_S1_L001_R2_001.fastq.gz This is shovil 0.9.0 Written by Torsten Seemann <torsten.seemann@gmail.com> Homepage is https://github.com/tseemann/shovill Operating system is darwin Using seqtk - /usr/local/bin/seqtk Using pigz - /usr/local/bin/pigz Using kmc - /usr/local/bin/kmc Using trimmomatic - /usr/local/bin/trimmomatic Using lighter - /usr/local/bin/lighter Using flash - /usr/local/bin/flash Using spades.py - /usr/local/bin/spades.py Using bwa - /usr/local/bin/bwa Using samtools - /usr/local/bin/samtools Using java - /usr/bin/java Using pilon - /usr/local/bin/pilon Using tee - /usr/bin/tee Using sysctl - /usr/sbin/sysctl Machine has 24 CPU cores and 64 GB RAM Changing into folder: /Users/xxx/Desktop/Clav04/shovill Collecting raw read statistics with 'seqtk' Running: seqtk fqchk -q3 \/Users\/BrodersLab\/Desktop\/Clav04\/Clav04_S1_L001_R1_001\.fastq\.gz > 10-seqtk.tab Read stats: total_bp = 433406240 Read stats: max_len = 151 Read stats: min_len = 151 Read stats: avg_len = 151 Estimating genome size with 'kmc' Running: kmc -m8 -sm -n256 -ci3 -k25 -t16 \/Users\/xxx\/Desktop\/Clav04\/Clav04_S1_L001_R1_001\.fastq\.gz kmc /var/folders/mp/rjltg4g177v1jqwywpw2gjn00000gn/T/PJj2WG5Olm >> 20-kmc.log 2>&1 Error 256 running command

Here you can see the 20-kmc.log file:

********************************* Error: Cannot open temporary file /var/folders/mp/rjltg4g177v1jqwywpw2gjn00000gn/T/PJj2WG5Olm/kmc_00253.bin

[tseemann]

@giriarteS

what does it say in shovel/20-kmc.log ?

are you using shovill 0.9 ?
there was a kmc bug in 0.8 release

what version of kmc are you using?

% kmc
K-Mer Counter (KMC) ver. 3.0.0 (2017-01-28)

[tseemann]

@stephenturner can you send me the logs? did you use the --ram option ?

[giriarteS]

@tseeman, I am using shovill 0.9 and kmc Version 3.0.0 (2017-01-28).

[MohamedRefaat92]

I'm having the same problem. I downloaded shovill yesterday and it worked fine on different samples. Today, when I executed the same command on the same data, I get this error after java :

Running: _JAVA_OPTIONS=-Xmx8g pilon --genome contigs.fasta --frags shovill.bam --fix bases --output pilon --threads 1 --changes --mindepth 0.25 >> 80-pilon.log 2>&1
Error 256 running command

This is the command I used. shovill --outdir mrsa --R1 1_S1_L001_R1_001.fastq.gz --R2 1_S1_L001_R2_001.fastq.gz I have to say that I tried to fix the problem by uninstalling then reinstalling shovill and it's dependencies, but I still get the same error.

[tseemann]

@stephenturner your error is Message 'Cannot open temporary file to write'. in shovill --help what is your --tmpdir set to? and does it have any space? This folder gets passed to spades. You also have some wierd linker errors with conda spades? Does it fail on the binaries you download from the Spades site?

[tseemann]

@MohamedRefaat92 Can you show me what the following command say?

• pilon --version
• which pilon
• head -n3 $(which pilon)
• java -version

[MohamedRefaat92]

$ pilon --version Pilon version 1.22 Wed Mar 15 16:38:30 2017 -0400

$which pilon /home/linuxbrew/.linuxbrew/bin/pilon

$head -n3 $(which pilon)

#!/bin/bash
exec java -Xmx1000m -Xms20m -jar /home/linuxbrew/.linuxbrew/Cellar/pilon/1.22/pilon-1.22.jar "$@"

$ java -version

openjdk version "1.8.0_162"
OpenJDK Runtime Environment (build 1.8.0_162-8u162-b12-0ubuntu0.16.04.2-b12)
OpenJDK 64-Bit Server VM (build 25.162-b12, mixed mode)

[tseemann]

@MohamedRefaat92 that all looks the same as mine. I am working on a new version of Shovill right now and will release in 6 hours or so. It will have much better output to help find the bug.

[giriarteS]

Below you can see the error message in shovel/20-kmc.log file:

Error: Cannot open temporary file /var/folders/mp/rjltg4g177v1jqwywpw2gjn00000gn/T/PJj2WG5Olm/kmc_00253.bin

I tried to run shovill again using the genome size option. I got a different error (a segmentation fault) with lighter.

[tseemann]

@giriarteS
Are you using a very old Mac computer? Or a very new one? How are you installing all the software? via Homebrew?

Operating system is darwin
Machine has 24 CPU cores and 64 GB RAM

[giriarteS]

I used homebrew to install shovill in a new Mac Pro.

[stephenturner]

My issue was the tmpdir. I defaulted to using /tmp, which only has a few gb available. Changing that solved my problem.

On Fri, May 4, 2018 at 9:31 PM Torsten Seemann notifications@github.com wrote:

@stephenturner https://github.com/stephenturner can you send me the logs? did you use the --ram option ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tseemann/shovill/issues/59#issuecomment-386769903, or mute the thread https://github.com/notifications/unsubscribe-auth/AAcFLONa5bNBLZb2oMP8W2IwsLDIexBHks5tvQEEgaJpZM4TAjBO .

[PlantDr430]

I started having a problem with Flash on one of my runs. I have already successfully completed 6 assemblies today using the same pipeline, however, this particular pair of reads is giving me trouble. I keep making it to Flash and get the 256 running command error a little after flash starts. I've tried increasing the ulimit, which allowed Flash to run slightly longer with this set, but ulimit is set to 8,192 and I can't increase it any higher.

The Flash error is below and is in the log file attached. I'm just curious as to why this particular set of reads is failing when I have already done multiple runs today.

"Error writing "./flash.extendedFrags.fastq.gz"[FLASH] FLASH did not complete successfully; exiting with failure status (1)

Here is my run:

shovill --minlen 1000 --cpus 24 --depth 95 --outdir LM71_shovill_output --R1 LM71J_2_S2_R1_001.fasta.gz --R2 LM71J_2_S2_R2_001.fastq.gz

Homepage is https://github.com/tseemann/shovill Operating system is linux Using seqtk - /usr/bin/seqtk Using pigz - /usr/bin/pigz Using kmc - /usr/bin/kmc Using trimmomatic - /usr/bin/trimmomatic Using lighter - /usr/bin/lighter Using flash - /usr/bin/flash Using spades.py - /usr/bin/spades.py Using bwa - /usr/bin/bwa Using samtools - /usr/local/bin/samtools Using java - /usr/bin/java Using pilon - /usr/bin/pilon Using tee - /usr/bin/tee Using vmstat - /usr/bin/vmstat Machine has 24 CPU cores and 251.900695800781 GB RAM Changing into folder: /home/stephenwyka/ClavicepsGenomes/MiaoGenomes/LM71_shovill_output Collecting raw read statistics with 'seqtk' Running: seqtk fqchk -q3 \/home\/stephenwyka\/ClavicepsGenomes\/MiaoGenomes\/LM71J_2_S2_R1_001.fastq.gz > 10-seqtk.tab Read stats: min_len = 35 Read stats: total_bp = 6272613692 Read stats: avg_len = 148 Read stats: max_len = 151 Estimating genome size with 'kmc' Running: kmc -m8 -sm -n256 -ci3 -k25 -t24 \/home\/stephenwyka\/ClavicepsGenomes\/MiaoGenomes\/LM71J_2_S2_R1_001.fastq.gz kmc /tmp/0QLBjT60lN >> 20-kmc.log 2>&1 Using genome size 34235164 bp Estimated sequencing depth: 183 x Subsampling reads by factor 0.519 to get from 183x to 95x Running: seqtk sample \/home\/stephenwyka\/ClavicepsGenomes\/MiaoGenomes\/LM71J_2_S2_R1_001.fastq.gz 0.519 | pigz --fast -c -p 24 > R1.sub.fq.gz Running: seqtk sample \/home\/stephenwyka\/ClavicepsGenomes\/MiaoGenomes\/LM71J_2_S2_R2_001.fastq.gz 0.519 | pigz --fast -c -p 24 > R2.sub.fq.gz Running: ln -s R1.sub.fq.gz R1.fq.gz Running: ln -s R2.sub.fq.gz R2.fq.gz Average read length looks like 148 bp Setting k-mer range to (31 .. 111) Estimated K-mers: 31 51 71 91 111 [kn=5, ks=20, kmin=31, kmax=111] Using kmers: 31,51,71,91,111 Correcting reads with 'Lighter' Running: lighter -od . -r R1.fq.gz -r R2.fq.gz -K 32 34235164 -t 24 -maxcor 1 >> 40-lighter.log 2>&1 Overlapping reads with 'FLASH' Running: flash -m 20 -M 151 -d . -o flash -z -t 24 R1.cor.fq.gz R2.cor.fq.gz >> 50-flash.log 2>&1 Error 256 running command

And the three available log files are here 20-kmc.log 40-lighter.log 50-flash.log

Thank you

[PlantDr430]

Nevermind, problem was resolved.

[tseemann]

@PlantDr430 thats good to hear! I have made the shovill.log MUCH better now.

[bifidoman]

Below you can see the error message in shovel/20-kmc.log file:

Error: Cannot open temporary file /var/folders/mp/rjltg4g177v1jqwywpw2gjn00000gn/T/PJj2WG5Olm/kmc_00253.bin

Hi,

just FYI I got the same error, and I noticed that the tmp folder did not exist. I created the folder and It worked. Shovill 0.9.0 on Mac Catalina best

marc

[tseemann]

@bifidoman lots of bug fixes since 0.9.0 - try and upgrade to 1.0.8 if you can!

[emmannaemeka]

I am having smilier issue

Using seqtk - /Users/emmannaemeka/miniconda3/bin/seqtk Using pigz - /Users/emmannaemeka/miniconda3/bin/pigz Using kmc - /Users/emmannaemeka/miniconda3/bin/kmc Using trimmomatic - /Users/emmannaemeka/miniconda3/bin/trimmomatic Using lighter - /Users/emmannaemeka/miniconda3/bin/lighter Using flash - /Users/emmannaemeka/miniconda3/bin/flash Using spades.py - /Users/emmannaemeka/miniconda3/bin/spades.py Using bwa - /Users/emmannaemeka/miniconda3/bin/bwa Using samtools - /usr/local/bin/samtools Using java - /Users/emmannaemeka/miniconda3/bin/java Using pilon - /Users/emmannaemeka/miniconda3/bin/pilon Using tee - /usr/bin/tee Using sysctl - /usr/sbin/sysctl Machine has 4 CPU cores and 16 GB RAM Forced removal of existing --outdir spo5_shovill (please wait) Changing into folder: /Users/emmannaemeka/Documents/Research/salmonella_phage_project/spo5/spo5_shovill Collecting raw read statistics with 'seqtk' Running: seqtk fqchk -q3 \/Users\/emmannaemeka\/Documents\/Research\/salmonella_phage_project\/spo5\/21-005-0894_R1.forward_paired.fq.gz > 10-seqtk.tab Read stats: min_len = 36 Read stats: max_len = 151 Read 20-kmc.log stats: total_bp = 385940918 Read stats: avg_len = 148 Estimating genome size with 'kmc' Running: kmc -m8 -sm -n256 -ci3 -k25 -t4 \/Users\/emmannaemeka\/Documents\/Research\/salmonella_phage_project\/spo5\/21-005-0894_R1.forward_paired.fq.gz kmc /Users/emmannaemeka/Documents/Research/salmonella_phage_project/spo5/W27fF0qrUD >> 20-kmc.log 2>&1 Error 256 running command

I tried to change temporary folder

shovill --outdir spo5_shovill --R1 21-005-0894_R1.forward_paired.fq.gz --R2 21-005-0894_R2.reverse_paired.fq.gz --tmpdir /Users/emmannaemeka/Documents/Research/salmonella_phage_project/spo5

When I ran less /var/folders/8v/08h9bb_s5jl6_f70nwzmj6jr0000gn/T/WSPCtMFcxb/kmc_00253.bin /var/folders/8v/08h9bb_s5jl6_f70nwzmj6jr0000gn/T/WSPCtMFcxb/kmc_00253.bin: No such file or directory

How can I solve this?