search

bxlab / metaWRAP

MetaWRAP - a flexible pipeline for genome-resolved metagenomic data analysis
MIT License
389 stars 189 forks source link

failure to download KRAKEN database #79

Open zhechen95 opened 5 years ago

zhechen95 commented 5 years ago

hi， when i am downloading the KRAKEN database ，meeting the question that Found jellyfish v1.1.12 kmer_estimator: thread count exceeds number of processors . so the kraken database is incomplete ,i have know my processors is 16 and can't be HT,so the threads are not enough , then i change the command the number "24" of kraken-build --standard --threads 24 --db MY_KRAKEN_DATABASE to 16 . but it does't work as well . the warming is

 Found jellyfish v1.1.12
Skipping archaea, already completed library download
Skipping bacteria, already completed library download
Skipping viral, already completed library download
Kraken build set to minimize disk writes.
Creating k-mer set (step 1 of 6)...
Found jellyfish v1.1.12
Hash size not specified, using '25106654720'
terminate called after throwing an instance of 'jellyfish::invertible_hash::ErrorAllocation'
  what():  Failed to allocate 181169529640 bytes of memory
xargs: cat: terminated by signal 13
/home/chenzhe/miniconda3/envs/metawrap-env/libexec/build_kraken_db.sh: line 96:  4827 Done                    find library/ -name '*.fna' -print0
      4828 Exit 125                | xargs -0 cat
      4829 Aborted                 (core dumped) | jellyfish count -m $KRAKEN_KMER_LEN -s $KRAKEN_HASH_SIZE -C -t $KRAKEN_THREAD_CT -o database /dev/fd/0

in addition to this i also confused with the command KRAKEN_DB=/path/to/my/database/MY_KRAKEN_DATABASE” it must have done？ what is specific mean？

i am a freshman about this ,so i really really hope you can help me with this problem ,i will appreciate you a lot. good day

ursky commented 5 years ago

The error you are getting now means that you do not have enough RAM to build the Kraken database. I would look into MiniKraken databases on their website and see if there is a solution that has a smaller memory footprint: https://ccb.jhu.edu/software/kraken/. Unfortunately, I would not be able to help with that. And the Kraken database configuration needs to be done in the config-metawrap file that was installed with metawrap. Run which config-metawrap to find it. Although if you can't build a Kraken database, then you still cant use the Kraken module.

zhechen95 commented 5 years ago

thanks for your reply i am sorry i haven't solve this problem ，so does this vision of kraken is kraken 2 ，can i download standard database rather than minikraken database from the website directly ？ i tried but i failed

kunstner commented 5 years ago

Hi @ursky, I have a related question. Can I just download the minikraken DB and place it in the folder specified in config-metawrap to be able to run the kraken module? Is it possible to even use minikraken2 for the analysis? Best, Axel

ursky commented 5 years ago

You should be able to use the minikraken DB with metawrap, although I have not tested. Minikraken2 is for kraken2 only, however. If RAM is limiting, a simpler solution might be just passing the --no-preload option with metawrap's kraken module, which results in a much smaller memory footprint.

kunstner commented 5 years ago

Thanks for your reply @ursky