search

soedinglab / metaeuk

MetaEuk - sensitive, high-throughput gene discovery and annotation for large-scale eukaryotic metagenomics
GNU General Public License v3.0
174 stars 23 forks source link

Can not allocate entries memory in IndexTable::initMemory #3

Closed sarah872 closed 4 years ago

sarah872 commented 4 years ago

I am running the pre-compiled version of Metaeuk, but I have an issue that is (I think) related to the available memory.

This is the log:

createdb /tmp/slurm-6704152/sequence.fasta sequence --dbtype 2

MMseqs Version:         3dcf1f12dd7e0ac64af47a0d0f995a9c0ceb5200
Max sequence length     65535
Database type           2
Shuffle input database  true
Createdb mode           0
Offset of numeric ids   0
Compressed              0
Verbosity               3

Converting sequences
[===================
Time for merging to sequence_h: 0h 0m 0s 530ms
Time for merging to sequence: 0h 0m 0s 209ms
Time for merging to sequence.lookup: 0h 0m 0s 0ms
Time for processing: 0h 0m 2s 147ms
extractorfs /tmp/slurm-6704152/sequence /tmp/slurm-6704152/temp/8679630176196528148/nucl_6f --min-length 15 --max-length 32734 --max-gaps 2147483647 --contig-start-mode 2 --contig-end-mode 2 --orf-start-mode 1 --forward-frames 1,2,3 --reverse-frames 1,2,3 --translation-table 1 --translate 0 --use-all-table-starts 0 --id-offset 0 --create-lookup 0 --threads 20 --compressed 0 -v 3

[=================================================================] 199.11K 1s 302ms
Time for merging to nucl_6f_h: 0h 0m 1s 147ms
Time for merging to nucl_6f: 0h 0m 1s 916ms
Time for processing: 0h 0m 6s 393ms
translatenucs /tmp/slurm-6704152/temp/8679630176196528148/nucl_6f /tmp/slurm-6704152/temp/8679630176196528148/aa_6f --translation-table 1 --add-orf-stop 0 -v 3 --compressed 0 --threads 20

[=================================================================] 3.91M 2s 45ms
Time for merging to aa_6f: 0h 0m 2s 31ms
Time for processing: 0h 0m 4s 745ms
Tmp /tmp/slurm-6704152/temp/8679630176196528148/tmp_search folder does not exist or is not a directory.
prefilter /tmp/slurm-6704152/temp/8679630176196528148/aa_6f /tmp/slurm-6704152/database/MMETSP_uniclust50_MERC_profiles /tmp/slurm-6704152/temp/8679630176196528148/tmp_search/13546197791315708787/pref --sub-mat nucl:nucleotide.out,aa:blosum62.out --seed-sub-mat nucl:nucleotide.out,aa:VTML80.out -s 4 -k 0 --k-score 2147483647 --alph-size 21 --max-seq-len 65535 --max-seqs 300 --split 0 --split-mode 2 --split-memory-limit 0 -c 0 --cov-mode 0 --comp-bias-corr 1 --diag-score 1 --exact-kmer-matching 0 --mask 1 --mask-lower-case 0 --min-ungapped-score 15 --add-self-matches 0 --spaced-kmer-mode 1 --db-load-mode 0 --pca 1 --pcb 1.5 --threads 20 --compressed 0 -v 3

Query database size: 3909404 type: Aminoacid
Estimated memory consumption: 181G
Target database size: 87984812 type: Profile
Index table k-mer threshold: 132 at k-mer size 7
Index table: counting k-mers
[=================================================================] 87.98M 68h 56m 59s 285ms
Index table: Masked residues: 0
Can not allocate entries memory in IndexTable::initMemory
Error: Prefilter died
Error: search step died
milot-mirdita commented 4 years ago

You need to pass it the --slice-search parameter or it will use the wrong profile search mode. The default profile search mode only works well for a few hundred thousand profiles and not for the huge profile database we provide for download.

If you want a set of stickers (see https://twitter.com/thesteinegger/status/1201076220957315074), send me your address to milot at mirdita de.