GuoliangLi-HZAU / BatMeth2

BS-seq analysis pipeline
38 stars 12 forks source link

FM index load error on TAIR10.fa #13

Closed MrDigest0305 closed 4 years ago

MrDigest0305 commented 4 years ago

Here is the command and the log:

[daijie20171011@compute01 index]$ BatMeth2 build_index TAIR10.fa

/gss1/home/daijie20171011/batmeth2/index [ Program directory ] /gss1/home/daijie20171011/biosoft/BatMeth2/bin/ [ Program name ] BatMeth2 [ Workdir ] /gss1/home/daijie20171011/batmeth2/index /gss1/home/daijie20171011/biosoft/BatMeth2/bin/build_all TAIR10.fa -= Index Builder for Batman =- Building on /gss1/home/daijie20171011/batmeth2/index Stripping TAIR10.fa and filtering nucleotides... Converting C->T Creating FASTA file of Reverse of the Genome... Creating FM index of reverse genome... BWTFormatdb v1.0, Copyright (C) 2006, Wong Chi Kwong. BWTFormatdb comes with ABSOLUTELY NO WARRENTY. BWTFormatdb is free software, and you are welcome to redistribute it under certain conditions. For details type BWTFormatdb.

Loading /gss1/home/daijie20171011/biosoft/BatMeth2/bin/bwtformatdb.ini ..done.

Parse FASTA file : Y Build BWT : Y Build SA value : Y Build SA index : Y

Show progress : Y

Parse FASTA : Mask lower case : N Random seed : 1567503216

Build BWT : Target N Bits : 2.50 Occ value frequency : 256 Initial Max Build Size : 10000000 Inc Max Build Size : 10000000

Build SA value : SA value frequency : 8

Build SA index : SA index no. of char : 12

Annotation file : ./revTAIR10.fa-CtoT.ann Ambigurity file : ./revTAIR10.fa-CtoT.amb Packed DNA file : ./revTAIR10.fa-CtoT.pac BWT Code file : ./revTAIR10.fa-CtoT.bwt BWT Occ value file : ./revTAIR10.fa-CtoT.fmv SA value file : ./revTAIR10.fa-CtoT.sa Cached SA index file : ./revTAIR10.fa-CtoT.sai

Parsing FASTA file.. Finished. Parsed 1 sequences. Elapsed time = 1.92 s

Building BWT.. 10 iterations done. 29216502 characters processed. 20 iterations done. 51116198 characters processed. 30 iterations done. 68388198 characters processed. 40 iterations done. 82009526 characters processed. 50 iterations done. 92751014 characters processed. 60 iterations done. 101220726 characters processed. 70 iterations done. 107898326 characters processed. 80 iterations done. 113162150 characters processed. 90 iterations done. 117310742 characters processed. Finished constructing BWT in 97 iterations. Elapsed time = 23.80 s

Saving BWT.. Finished saving BWT. Elapsed time = 0.02 s

Loading BWT... Finished loading BWT. Elapsed time = 0.07 s

Building SA value... SA Value generated : 1495846 SA Value generated : 2991692 SA Value generated : 4487538 SA Value generated : 5983384 SA Value generated : 7479230 SA Value generated : 8975076 SA Value generated : 10470922 SA Value generated : 11966768 SA Value generated : 13462614 SA Value generated : 14958460 SA Value generated : 14958469 Finished building SA value. Elapsed time = 8.57 s

Building cached SA index... Finished building cached SA index. Elapsed time = 0.26 s

Finished all tasks. Total elapsed time = 34.64 s

Maximum amount of memory allocated: 162987932 Maximum amount of memory dispatched: 161182768 Number of char : 119667750 Bit per char : 10.78

Creating FM index of the genome... BWTFormatdb v1.0, Copyright (C) 2006, Wong Chi Kwong. BWTFormatdb comes with ABSOLUTELY NO WARRENTY. BWTFormatdb is free software, and you are welcome to redistribute it under certain conditions. For details type BWTFormatdb.

Loading /gss1/home/daijie20171011/biosoft/BatMeth2/bin/bwtformatdb.ini ..done.

Parse FASTA file : Y Build BWT : Y Build SA value : Y Build SA index : Y

Show progress : Y

Parse FASTA : Mask lower case : N Random seed : 1567503251

Build BWT : Target N Bits : 2.50 Occ value frequency : 256 Initial Max Build Size : 10000000 Inc Max Build Size : 10000000

Build SA value : SA value frequency : 8

Build SA index : SA index no. of char : 12

Annotation file : TAIR10.fa-CtoT.non.ann Ambigurity file : TAIR10.fa-CtoT.non.amb Packed DNA file : TAIR10.fa-CtoT.non.pac BWT Code file : TAIR10.fa-CtoT.non.bwt BWT Occ value file : TAIR10.fa-CtoT.non.fmv SA value file : TAIR10.fa-CtoT.non.sa Cached SA index file : TAIR10.fa-CtoT.non.sai

Parsing FASTA file.. Finished. Parsed 7 sequences. Elapsed time = 2.04 s

Building BWT.. 10 iterations done. 29586359 characters processed. 20 iterations done. 51763335 characters processed. 30 iterations done. 69254039 characters processed. 40 iterations done. 83047927 characters processed. 50 iterations done. 93925543 characters processed. 60 iterations done. 102502631 characters processed. 70 iterations done. 109264919 characters processed. 80 iterations done. 114595543 characters processed. 90 iterations done. 118796855 characters processed. Finished constructing BWT in 97 iterations. Elapsed time = 23.91 s

Saving BWT.. Finished saving BWT. Elapsed time = 0.02 s

Loading BWT... Finished loading BWT. Elapsed time = 0.06 s

Building SA value... SA Value generated : 1514781 SA Value generated : 3029562 SA Value generated : 4544343 SA Value generated : 6059124 SA Value generated : 7573905 SA Value generated : 9088686 SA Value generated : 10603467 SA Value generated : 12118248 SA Value generated : 13633029 SA Value generated : 15147810 SA Value generated : 15147817 Finished building SA value. Elapsed time = 8.96 s

Building cached SA index... Finished building cached SA index. Elapsed time = 0.33 s

Finished all tasks. Total elapsed time = 35.33 s

Maximum amount of memory allocated: 164171740 Maximum amount of memory dispatched: 162366960 Number of char : 121182535 Bit per char : 10.72

Deleting auxilliary files... FM index load error -= Index Builder for Batman =- Building on /gss1/home/daijie20171011/batmeth2/index Stripping TAIR10.fa and filtering nucleotides... Converting G->A Creating FASTA file of Reverse of the Genome... Creating FM index of reverse genome... BWTFormatdb v1.0, Copyright (C) 2006, Wong Chi Kwong. BWTFormatdb comes with ABSOLUTELY NO WARRENTY. BWTFormatdb is free software, and you are welcome to redistribute it under certain conditions. For details type BWTFormatdb.

Loading /gss1/home/daijie20171011/biosoft/BatMeth2/bin/bwtformatdb.ini ..done.

Parse FASTA file : Y Build BWT : Y Build SA value : Y Build SA index : Y

Show progress : Y

Parse FASTA : Mask lower case : N Random seed : 1567503288

Build BWT : Target N Bits : 2.50 Occ value frequency : 256 Initial Max Build Size : 10000000 Inc Max Build Size : 10000000

Build SA value : SA value frequency : 8

Build SA index : SA index no. of char : 12

Annotation file : ./revTAIR10.fa-GtoA.ann Ambigurity file : ./revTAIR10.fa-GtoA.amb Packed DNA file : ./revTAIR10.fa-GtoA.pac BWT Code file : ./revTAIR10.fa-GtoA.bwt BWT Occ value file : ./revTAIR10.fa-GtoA.fmv SA value file : ./revTAIR10.fa-GtoA.sa Cached SA index file : ./revTAIR10.fa-GtoA.sai

Parsing FASTA file.. Finished. Parsed 1 sequences. Elapsed time = 1.69 s

Building BWT.. 10 iterations done. 29216502 characters processed. 20 iterations done. 51116198 characters processed. 30 iterations done. 68388198 characters processed. 40 iterations done. 82009526 characters processed. 50 iterations done. 92751014 characters processed. 60 iterations done. 101220726 characters processed. 70 iterations done. 107898326 characters processed. 80 iterations done. 113162150 characters processed. 90 iterations done. 117310742 characters processed. Finished constructing BWT in 97 iterations. Elapsed time = 23.11 s

Saving BWT.. Finished saving BWT. Elapsed time = 0.03 s

Loading BWT... Finished loading BWT. Elapsed time = 0.08 s

Building SA value... SA Value generated : 1495846 SA Value generated : 2991692 SA Value generated : 4487538 SA Value generated : 5983384 SA Value generated : 7479230 SA Value generated : 8975076 SA Value generated : 10470922 SA Value generated : 11966768 SA Value generated : 13462614 SA Value generated : 14958460 SA Value generated : 14958469 Finished building SA value. Elapsed time = 8.11 s

Building cached SA index... Finished building cached SA index. Elapsed time = 0.18 s

Finished all tasks. Total elapsed time = 33.19 s

Maximum amount of memory allocated: 162987932 Maximum amount of memory dispatched: 161182768 Number of char : 119667750 Bit per char : 10.78

Creating FM index of the genome... BWTFormatdb v1.0, Copyright (C) 2006, Wong Chi Kwong. BWTFormatdb comes with ABSOLUTELY NO WARRENTY. BWTFormatdb is free software, and you are welcome to redistribute it under certain conditions. For details type BWTFormatdb.

Loading /gss1/home/daijie20171011/biosoft/BatMeth2/bin/bwtformatdb.ini ..done.

Parse FASTA file : Y Build BWT : Y Build SA value : Y Build SA index : Y

Show progress : Y

Parse FASTA : Mask lower case : N Random seed : 1567503321

Build BWT : Target N Bits : 2.50 Occ value frequency : 256 Initial Max Build Size : 10000000 Inc Max Build Size : 10000000

Build SA value : SA value frequency : 8

Build SA index : SA index no. of char : 12

Annotation file : TAIR10.fa-GtoA.non.ann Ambigurity file : TAIR10.fa-GtoA.non.amb Packed DNA file : TAIR10.fa-GtoA.non.pac BWT Code file : TAIR10.fa-GtoA.non.bwt BWT Occ value file : TAIR10.fa-GtoA.non.fmv SA value file : TAIR10.fa-GtoA.non.sa Cached SA index file : TAIR10.fa-GtoA.non.sai

Parsing FASTA file.. Finished. Parsed 7 sequences. Elapsed time = 2.07 s

Building BWT.. 10 iterations done. 29586359 characters processed. 20 iterations done. 51763335 characters processed. 30 iterations done. 69254039 characters processed. 40 iterations done. 83047927 characters processed. 50 iterations done. 93925543 characters processed. 60 iterations done. 102502631 characters processed. 70 iterations done. 109264919 characters processed. 80 iterations done. 114595543 characters processed. 90 iterations done. 118796855 characters processed. Finished constructing BWT in 97 iterations. Elapsed time = 24.35 s

Saving BWT.. Finished saving BWT. Elapsed time = 0.02 s

Loading BWT... Finished loading BWT. Elapsed time = 0.08 s

Building SA value... SA Value generated : 1514781 SA Value generated : 3029562 SA Value generated : 4544343 SA Value generated : 6059124 SA Value generated : 7573905 SA Value generated : 9088686 SA Value generated : 10603467 SA Value generated : 12118248 SA Value generated : 13633029 SA Value generated : 15147810 SA Value generated : 15147817 Finished building SA value. Elapsed time = 8.23 s

Building cached SA index... Finished building cached SA index. Elapsed time = 0.27 s

Finished all tasks. Total elapsed time = 35.02 s

Maximum amount of memory allocated: 164171740 Maximum amount of memory dispatched: 162366960 Number of char : 121182535 Bit per char : 10.72

Deleting auxilliary files... FM index load error -= Index Builder for Batman =- Building on /gss1/home/daijie20171011/batmeth2/index Stripping TAIR10.fa and filtering nucleotides... Creating FASTA file of Reverse of the Genome... Creating FM index of reverse genome... BWTFormatdb v1.0, Copyright (C) 2006, Wong Chi Kwong. BWTFormatdb comes with ABSOLUTELY NO WARRENTY. BWTFormatdb is free software, and you are welcome to redistribute it under certain conditions. For details type BWTFormatdb.

Loading /gss1/home/daijie20171011/biosoft/BatMeth2/bin/bwtformatdb.ini ..done.

Parse FASTA file : Y Build BWT : Y Build SA value : Y Build SA index : Y

Show progress : Y

Parse FASTA : Mask lower case : N Random seed : 1567503359

Build BWT : Target N Bits : 2.50 Occ value frequency : 256 Initial Max Build Size : 10000000 Inc Max Build Size : 10000000

Build SA value : SA value frequency : 8

Build SA index : SA index no. of char : 12

Annotation file : ./revTAIR10.fa.ann Ambigurity file : ./revTAIR10.fa.amb Packed DNA file : ./revTAIR10.fa.pac BWT Code file : ./revTAIR10.fa.bwt BWT Occ value file : ./revTAIR10.fa.fmv SA value file : ./revTAIR10.fa.sa Cached SA index file : ./revTAIR10.fa.sai

Parsing FASTA file.. Finished. Parsed 1 sequences. Elapsed time = 2.34 s

Building BWT.. 10 iterations done. 29216502 characters processed. 20 iterations done. 51116198 characters processed. 30 iterations done. 68388198 characters processed. 40 iterations done. 82009526 characters processed. 50 iterations done. 92751014 characters processed. 60 iterations done. 101220726 characters processed. 70 iterations done. 107898326 characters processed. 80 iterations done. 113162150 characters processed. 90 iterations done. 117310742 characters processed. Finished constructing BWT in 97 iterations. Elapsed time = 25.03 s

Saving BWT.. Finished saving BWT. Elapsed time = 0.02 s

Loading BWT... Finished loading BWT. Elapsed time = 0.12 s

Building SA value... SA Value generated : 1495846 SA Value generated : 2991692 SA Value generated : 4487538 SA Value generated : 5983384 SA Value generated : 7479230 SA Value generated : 8975076 SA Value generated : 10470922 SA Value generated : 11966768 SA Value generated : 13462614 SA Value generated : 14958460 SA Value generated : 14958469 Finished building SA value. Elapsed time = 10.90 s

Building cached SA index... Finished building cached SA index. Elapsed time = 1.88 s

Finished all tasks. Total elapsed time = 40.29 s

Maximum amount of memory allocated: 162987932 Maximum amount of memory dispatched: 161182768 Number of char : 119667750 Bit per char : 10.78

Creating FM index of the genome... BWTFormatdb v1.0, Copyright (C) 2006, Wong Chi Kwong. BWTFormatdb comes with ABSOLUTELY NO WARRENTY. BWTFormatdb is free software, and you are welcome to redistribute it under certain conditions. For details type BWTFormatdb.

Loading /gss1/home/daijie20171011/biosoft/BatMeth2/bin/bwtformatdb.ini ..done.

Parse FASTA file : Y Build BWT : Y Build SA value : Y Build SA index : Y

Show progress : Y

Parse FASTA : Mask lower case : N Random seed : 1567503399

Build BWT : Target N Bits : 2.50 Occ value frequency : 256 Initial Max Build Size : 10000000 Inc Max Build Size : 10000000

Build SA value : SA value frequency : 8

Build SA index : SA index no. of char : 12

Annotation file : TAIR10.fa.non.ann Ambigurity file : TAIR10.fa.non.amb Packed DNA file : TAIR10.fa.non.pac BWT Code file : TAIR10.fa.non.bwt BWT Occ value file : TAIR10.fa.non.fmv SA value file : TAIR10.fa.non.sa Cached SA index file : TAIR10.fa.non.sai

Parsing FASTA file.. Finished. Parsed 7 sequences. Elapsed time = 1.94 s

Building BWT.. 10 iterations done. 29586359 characters processed. 20 iterations done. 51763335 characters processed. 30 iterations done. 69254039 characters processed. 40 iterations done. 83047927 characters processed. 50 iterations done. 93925543 characters processed. 60 iterations done. 102502631 characters processed. 70 iterations done. 109264919 characters processed. 80 iterations done. 114595543 characters processed. 90 iterations done. 118796855 characters processed. Finished constructing BWT in 97 iterations. Elapsed time = 27.18 s

Saving BWT.. Finished saving BWT. Elapsed time = 0.02 s

Loading BWT... Finished loading BWT. Elapsed time = 0.06 s

Building SA value... SA Value generated : 1514781 SA Value generated : 3029562 SA Value generated : 4544343 SA Value generated : 6059124 SA Value generated : 7573905 SA Value generated : 9088686 SA Value generated : 10603467 SA Value generated : 12118248 SA Value generated : 13633029 SA Value generated : 15147810 SA Value generated : 15147817 Finished building SA value. Elapsed time = 8.14 s

Building cached SA index... Finished building cached SA index. Elapsed time = 1.83 s

Finished all tasks. Total elapsed time = 39.18 s

Maximum amount of memory allocated: 164171740 Maximum amount of memory dispatched: 162366960 Number of char : 121182535 Bit per char : 10.72

Deleting auxilliary files... FM index load error

GuoliangLi-HZAU commented 4 years ago

Hi, I downloaded the tair10 genome from ftp://ftp.arabidopsis.org/home/tair/Sequences/whole_chromosomes/. The program ran without any errors. Can you provide your genome files or download links?

MrDigest0305 commented 4 years ago

I have sent the tair10 genome file that I used by 126 e-mail. Thanks a lot.

GuoliangLi-HZAU commented 4 years ago

Hi, we haven't received your email yet. Could you send the genome file to 1010170266@qq.com mailbox?

MrDigest0305 commented 4 years ago

I have sent the TAIR10 fasta file by 126 mailbox. Maybe my last e-mail was judeged as a trash by guoliang.li@mail.hzau.edu.cn mailbox.

MrDigest0305 commented 4 years ago

Hi, Dr.Li, I find that there is '\M' in my fasta which should be removed by 'dos2unix' and I have solved the problem.