search

DerrickWood / kraken2

The second version of the Kraken taxonomic sequence classification system
MIT License
727 stars 273 forks source link

Kraken2 custom db #423

Open ntromas opened 3 years ago

ntromas commented 3 years ago

Hi,

I got this error when I tried to generate a custom db:

Creating sequence ID to taxonomy ID map (step 1)... Sequence ID to taxonomy ID map already present, skipping map creation. Estimating required capacity (step 2)... Estimated hash table requirement: 102619470992 bytes Capacity estimation complete. [48m24.282s] Building database files (step 3)... Taxonomy parsed and converted. xargs: cat: terminated by signal 13 /home/nico/miniconda3/envs/kraken2/libexec/build_kraken2_db.sh: line 143: 1911963 Done list_sequence_files 1911964 Exit 125 | xargs -0 cat 1911965 Killed | build_db -k $KRAKEN2_KMER_LEN -l $KRAKEN2_MINIMIZER_LEN -S $KRAKEN2_SEED_TEMPLATE $KRAKEN2XFLAG -H hash.k2d.tmp -t taxo.k2d.tmp -o opts.k2d.tmp -n taxonomy/ -m $seqid2taxid_map_file -c $required_capacity -p $KRAKEN2_THREAD_CT $max_db_flag -B $KRAKEN2_BLOCK_SIZE -b $KRAKEN2_SUBBLOCK_SIZE -r $KRAKEN2_MIN_TAXID_BITS $fast_build_flag

Any idea to fix that?

Thanks!

Roopahk90 commented 3 years ago

Hi,

I am also having the same issue with the kraken2 custom database build. It is taking a long time I have been running this for four days. The files under the custom database last modified date are not updated either - it's two days older.

I am running step 3, building an 'nt' custom database with 72 threads and it is processing very slowly, following is the last line of the console output. Processed 2201114 sequences (11148201205 bp)...

@jenniferlu717, @DerrickWood Can you pls suggest what is happening?

Thanks

xpli2020 commented 3 years ago

Hi,

I got this error when I tried to generate a custom db:

Creating sequence ID to taxonomy ID map (step 1)... Sequence ID to taxonomy ID map already present, skipping map creation. Estimating required capacity (step 2)... Estimated hash table requirement: 102619470992 bytes Capacity estimation complete. [48m24.282s] Building database files (step 3)... Taxonomy parsed and converted. xargs: cat: terminated by signal 13 /home/nico/miniconda3/envs/kraken2/libexec/build_kraken2_db.sh: line 143: 1911963 Done list_sequence_files 1911964 Exit 125 | xargs -0 cat 1911965 Killed | build_db -k $KRAKEN2_KMER_LEN -l $KRAKEN2_MINIMIZER_LEN -S $KRAKEN2_SEED_TEMPLATE $KRAKEN2XFLAG -H hash.k2d.tmp -t taxo.k2d.tmp -o opts.k2d.tmp -n taxonomy/ -m $seqid2taxid_map_file -c $required_capacity -p $KRAKEN2_THREAD_CT $max_db_flag -B $KRAKEN2_BLOCK_SIZE -b $KRAKEN2_SUBBLOCK_SIZE -r $KRAKEN2_MIN_TAXID_BITS $fast_build_flag

Any idea to fix that?

Thanks!

I think you have to check your memory, if not enough memory (in your case < 102619470992 bytes), it will kill the process. This happened to my computer. I had to swap memory.

xpli2020 commented 3 years ago

Hi,

I am also having the same issue with the kraken2 custom database build. It is taking a long time I have been running this for four days. The files under the custom database last modified date are not updated either - it's two days older.

I am running step 3, building an 'nt' custom database with 72 threads and it is processing very slowly, following is the last line of the console output. Processed 2201114 sequences (11148201205 bp)...

@jenniferlu717, @DerrickWood Can you pls suggest what is happening?

Thanks

Same issue with the nt database building. It reaches to a certain number of sequences then the number won't go up any more. Not sure if it is got stuck or something.

ntromas commented 3 years ago

Hi,

I have 128Go which should be enough to run this no? I am not sure the error message is really helpful here...

Cheers

lixiaopi1985 commented 3 years ago

Hi,

I have 128Go which should be enough to run this no? I am not sure the error message is really helpful here...

Cheers

Not for the NT, it showed it needed 240G at least as of 3/10/2021.

ntromas commented 3 years ago

Right but in my case it was required at least 102619470992 bytes...

Le jeu. 18 mars 2021 20 h 56, Xiaoping Li @.***> a écrit :

Hi,

I have 128Go which should be enough to run this no? I am not sure the error message is really helpful here...

Cheers

Not for the NT, it showed it needed 240G at least.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DerrickWood/kraken2/issues/423#issuecomment-802426013, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABY5D6GJOJNTJGQOPBCQUILTEKOLLANCNFSM4Y4SBYEQ .

lixiaopi1985 commented 3 years ago

Then I am not sure. Maybe it is a different issue after all.