aquaskyline / SOAPdenovo2

Next generation sequencing reads de novo assembler.
GNU General Public License v3.0
220 stars 78 forks source link

aio_return error #15

Closed dcopetti closed 7 years ago

dcopetti commented 8 years ago

Hello, I am running SOAPdn2 on a server and the log file contains the line "aio_return error" some millions of times. The log file is now of 796 GB, mostly these error lines. Above that, the lines are:

Version 2.04: released on July 13th, 2012
Compile Sep 16 2016     15:43:38
********************
Pregraph
********************
Parameters: pregraph -s bart_cfg_file -K 53 -p 20 -R -o bart_53 
In bart_cfg_file, 1 lib(s), maximum read length 120, maximum name length 256.
20 thread(s) initialized.
Import reads from file:
 /wing2/users/dcopetti/OGE_genomes_reads/new_datasets/bart_frag_SRR1528330_1.fastq
Import reads from file:
 /wing2/users/dcopetti/OGE_genomes_reads/new_datasets/bart_frag_SRR1528330_2.fastq
--- 100000000th reads.
--- 200000000th reads.
--- 300000000th reads.
--- 400000000th reads.

The server is using all the processors I requested and ~200 Gb of memory, I don't know if it is progressing. It is running like this since about 24 h, for a 400 Mb homozygous genome with some 50-60x of PE data. The command is SOAPdenovo-63mer all -s bart_cfg_file -p 20 -F -K 53 -R -o bart_53 1>bart_53_ass.log 2>bart_53_ass.err, the cfg file is

max_rd_len=120
[LIB]
#average insert size
avg_ins=400
#if sequence needs to be reversed
reverse_seq=0
asm_flags=3
rd_len_cutoff=120
rank=1
insert size)
pair_num_cutoff=3
least 32 for short insert size)
map_len=32
q1=/1.fastq
q2=/2.fastq

Is it all OK or is there something wrong? Thanks, Dario

aquaskyline commented 8 years ago

the problem is usually caused by file system error

Christina-hshi commented 7 years ago

I also got this error. Could you give more details about how this error happens? And what can I do to avoid or fix it? Thanks.

aquaskyline commented 7 years ago

@Christina-hshi could you please try install SOAPdenovo2 through bioconda or homebrew-science, they help you to take care of the library dependencies.

cchd0001 commented 7 years ago

@aquaskyline recently I also got this error . I want to debug it , but those functions that used to load file comes from libbam.a . So cound you please tell me where or who can I ask for those source codes ?

aquaskyline commented 7 years ago

@cchd0001 you can get libbam.a from samtools. the samtools version I used was 0.19.1

Christina-hshi commented 7 years ago

@aquaskyline I installed SOAPdenovo2 through both bioconda and homebrew-science. But for both cases, I still get the same error. @aquaskyline @cchd0001 Hope you can fix this problem soon. _

cchd0001 commented 7 years ago

@Christina-hshi Could you please check your input files? Make sure they are correct fastq/fasta format and correct gzip format if you used . I catch this issue twice last month , one was because the input file was invalid gzip format , another was because a garbage line at the end of input file . Also you can try https://github.com/cchd0001/SOAPdenovo2 . I think I just fixed this by report the incorrect data and exit the program .

Christina-hshi commented 7 years ago

@cchd0001 I checked my fasta file, the format should be fine. And I tried https://github.com/cchd0001/SOAPdenovo2, but I still got the aio_return error. Interestingly, I got rid of this error by removing four lines(or one read pair) from my fasta file (I tried to remove one read pair at either the beginning, the end or the middle). I can send you the fasta file if you want to take a look.

cchd0001 commented 7 years ago

@Christina-hshi yes , I want those files . thanks !

cchd0001 commented 7 years ago

@Christina-hshi Thanks for you data, I find a bug that if the size of input file is exactly N*32768, program will enter infinite loop with print "aio_return error". And this explain why you can avoid this bug by delete some data. The lastest code can run success with you data now. Hope this can help.

Christina-hshi commented 7 years ago

@cchd0001 Thanks. I tried the latest code, and it did work now.