search

smarco / gem3-mapper

GEM-Mapper v3
GNU General Public License v3.0
56 stars 17 forks source link

Maximum SA-bucket (6 GB) larger than max-memory-block allowed (3 GB #14

Closed sleepvet closed 4 years ago

sleepvet commented 4 years ago

I met a problem when I use command gem-indexer. The comment is "GEM::FatalError (sa_builder_store_suffixes.c:138,sa_builder_store_suffixes_prepare) SA Builder. Maximum SA-bucket (6 GB) larger than max-memory-block allowed (3 GB) ". How can I reset my max-memory-block allowed?

smarco commented 4 years ago

I'm sorry to hear that. To give you a better answer, I would need to know the total memory available on your computer, the reference size you are trying to index, as well as the cmd-line used to invoke the gem-indexer.

sleepvet commented 4 years ago

I'm sorry to hear that. To give you a better answer, I would need to know the total memory available on your computer, the reference size you are trying to index, as well as the cmd-line used to invoke the gem-indexer.

  1. memory inf: total used free shared buff/cache available Mem: 314G 7.0G 136G 57M 171G 305G Swap: 974M 221M 753M 2.ref size 4.3G
  2. cmd I used (the ref I used is protein seq, is that a problem?) gem-indexer -i rumiref100.faa -o rumiref100 4.comments from the GEM 2020/3/4 08:10:52 -- [Inspecting MultiFASTA] 2020/3/4 08:11:04 -- 100% ... done [12.130 s] 2020/3/4 08:11:04 -- Inspected text 6518951659 characters (index_complement=yes). Requesting 6216 MB (encoded text) 2020/3/4 08:11:04 -- [Reading MultiFASTA] 2020/3/4 08:11:05 -- 100000000 bases parsed 2020/3/4 08:11:06 -- 200000000 bases parsed 2020/3/4 08:11:07 -- 300000000 bases parsed 2020/3/4 08:11:08 -- 400000000 bases parsed 2020/3/4 08:11:10 -- 500000000 bases parsed 2020/3/4 08:11:11 -- 600000000 bases parsed 2020/3/4 08:11:12 -- 700000000 bases parsed 2020/3/4 08:11:13 -- 800000000 bases parsed 2020/3/4 08:11:14 -- 900000000 bases parsed 2020/3/4 08:11:16 -- 1000000000 bases parsed 2020/3/4 08:11:17 -- 1100000000 bases parsed 2020/3/4 08:11:18 -- 1200000000 bases parsed 2020/3/4 08:11:20 -- 1300000000 bases parsed 2020/3/4 08:11:21 -- 1400000000 bases parsed 2020/3/4 08:11:22 -- 1500000000 bases parsed 2020/3/4 08:11:24 -- 1600000000 bases parsed 2020/3/4 08:11:25 -- 1700000000 bases parsed 2020/3/4 08:11:26 -- 1800000000 bases parsed 2020/3/4 08:11:27 -- 1900000000 bases parsed 2020/3/4 08:11:28 -- 2000000000 bases parsed 2020/3/4 08:11:29 -- 2100000000 bases parsed 2020/3/4 08:11:31 -- 2200000000 bases parsed 2020/3/4 08:11:32 -- 2300000000 bases parsed 2020/3/4 08:11:34 -- 2400000000 bases parsed 2020/3/4 08:11:35 -- 2500000000 bases parsed 2020/3/4 08:11:36 -- 2600000000 bases parsed 2020/3/4 08:11:37 -- 2700000000 bases parsed 2020/3/4 08:11:39 -- 2800000000 bases parsed 2020/3/4 08:11:40 -- 2900000000 bases parsed 2020/3/4 08:11:41 -- 3000000000 bases parsed 2020/3/4 08:11:42 -- 3100000000 bases parsed 2020/3/4 08:11:43 -- 3200000000 bases parsed 2020/3/4 08:11:44 -- 3300000000 bases parsed 2020/3/4 08:11:44 -- Total 3307268360 bases parsed ...done [40.539 s] 2020/3/4 08:11:44 -- [Generating Text (explicit Reverse-Complement)] 2020/3/4 08:11:53 -- 100% ... done [8.920 s] 2020/3/4 08:12:07 -- [Generating BWT Forward-Text] 2020/3/4 08:12:07 -- [Building-BWT::Counting K-mers] 2020/3/4 08:12:28 -- 100% ... done [21.678 s] GEM::FatalError (sa_builder_store_suffixes.c:138,sa_builder_store_suffixes_prepare) SA Builder. Maximum SA-bucket (6 GB) larger than max-memory-block allowed (3 GB)
smarco commented 4 years ago

I'm afraid that the GEM-Mapper only allows for ACGTN characters. Otherwise, it will be encoded as an N. Perhaps this is causing excessive memory usage.

sleepvet commented 4 years ago

I'm afraid that the GEM-Mapper only allows for ACGTN characters. Otherwise, it will be encoded as an N. Perhaps this is causing excessive memory usage.

It's a pity that GEM mapper is not support mapping DNA reads back to protien references. It looks like I have to use the slower blastx. Thank you anyway, especially for your quick reply