Indexdb died error message when creating colabfold_envdb_202108_db with MMseqs

[gundalav]

Hi,

I was trying to setup the database. But it breaks upon the execution of this code:

mmseqs createindex colabfold_envdb_202108_db tmp2 --remove-tmp-files 1 The error message I get is this:

MMseqs Version:             edb8223d1ea07385ffe63d4f103af0eb12b2058e
Seed substitution matrix    aa:VTML80.out,nucl:nucleotide.out
k-mer length                0
Alphabet size               aa:21,nucl:5
Compositional bias          1
Max sequence length         65535
Max results per query       300
Mask residues               1
Mask lower case residues    0
Spaced k-mers               1
Spaced k-mer pattern
Sensitivity                 7.5
k-score                     seq:0,prof:0
Check compatible            0
Search type                 0
Split database              0
Split memory limit          0
Verbosity                   3
Threads                     8
Min codons in orf           30
Max codons in length        32734
Max orf gaps                2147483647
Contig start mode           2
Contig end mode             2
Orf start mode              1
Forward frames              1,2,3
Reverse frames              1,2,3
Translation table           1
Translate orf               0
Use all table starts        false
Offset of numeric ids       0
Create lookup               0
Compressed                  0
Add orf stop                false
Overlap between sequences   0
Sequence split mode         1
Header split mode           0
Strand selection            1
Remove temporary files      true

indexdb colabfold_envdb_202108_db colabfold_envdb_202108_db --seed-sub-mat aa:VTML80.out,nucl:nucleotide.out -k 0 --alph-size aa:21,nucl:5 --comp-bias-corr 1 --max-seq-len 65535 --max-seqs 300 --mask 1 --mask-lower-case 0 --spaced-kmer-mode 1 -s 7.5 --k-score seq:0,prof:0 --check-compatible 0 --search-type 0 --split 0 --split-memory-limit 0 -v 3 --threads 8

Target split mode. Searching through 34 splits
Estimated memory consumption: 29G
Write VERSION (0)
Write META (1)
Write SCOREMATRIX3MER (4)
Write SCOREMATRIX2MER (3)
Write SCOREMATRIXNAME (2)
Write SPACEDPATTERN (23)
Write GENERATOR (22)
Write DBR1INDEX (5)
Write DBR1DATA (6)
Write DBR2INDEX (7)
Killed
Error: indexdb died

It works fine with uniref30_2103.tar.gz file though.

How can I resolve the problem?

G.V.

[martin-steinegger]

I assume your computer does not have enough RAM. How much RAM does your server has?

[gundalav]

I am using AWS p3.2xlarge instance. It has around 61GB RAM.

[martin-steinegger]

Online searches: Our Colabfold server has ~760GB RAM and keeps full database and index in memory. Batch searches: To perform a batch search you require less memory. But its still approx 1 byte per residue. So I would assume you would probably require at least 90GB. We still need to figure out whats the lower bound for this database.

[maxshen29]

i have 128G RAM, but i have same erro . the erro : Estimated memory consumption: 560G Process needs more than 38G main memory. Increase the size of --split or set it to 0 to automatically optimize target database split. Write VERSION (0) Write META (1) Write SCOREMATRIX3MER (4) Write SCOREMATRIX2MER (3) Write SCOREMATRIXNAME (2) Write SPACEDPATTERN (23) Write GENERATOR (22) Write DBR1INDEX (5) Write DBR1DATA (6) Write DBR2INDEX (7) Write DBR2DATA (8) Write HDR1INDEX (18) Write HDR1DATA (19) Write ALNINDEX (24) Write ALNDATA (25) Index table: counting k-mers [=================================================================] 100.00% 209.34M 7m 34s 698ms
Index table: Masked residues: 1117805658 Can not allocate entries memory in IndexTable::initMemory Error: indexdb died