Building a microbial nt database

[DesmondoDekker]

Building a microbial nt database

Dear staff,

I am trying to build a microbial-nt database. According to the tutorial I run the following command:

krakenuniq-download --db DB --taxa "archaea,bacteria,viral,fungi,protozoa,helminths" --dust --exclude-environmental-taxa microbial-nt

Next I run the following command krakenuniq-build --db microbial-nt/ --kmer-len 31 --taxids-for-genomes --taxids-for-sequences

But I got the following error message.

_Kraken build set to minimize disk writes. Found 5 sequence files (*.{fna,fa,ffn,fasta,fsa}) in the library directory. Creating k-mer set (step 1 of 6)... Using /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish count_unique: malformed fasta file - expected header char > not found Hash size not specified, using '0' /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/builddb.sh: line 46: /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish: No such file or directory

Thanks a lot

Luigi

[alekseyzimin]

Please make sure you have installed krakenuniq with the option to install jellyfish ( -j ). Jellyfish v1.1 is mandatory for building databases.

[DesmondoDekker]

Dear Aleksey Zimin,

Thank I have installed jellyfish, but unfortunately, I get another error message using both the "--work-on-disk" and not "--work-on-disk option". Any clue about reducing the RAM required?.

Here the command:

!/bin/bash

SBATCH --partition=bioinfo

SBATCH --nodes=1

SBATCH --ntasks=1

SBATCH --cpus-per-task=12

SBATCH --mem=128000

source /opt/modules/init/bash module purge module load anaconda3 conda activate krakenuniq krakenuniq-build --db /data_agro/DB_microbe --kmer-len 31 --threads 12 --taxids-for-genomes --taxids-for-sequences --work-on-disk

---

• export VERBOSE=1
• VERBOSE=1
• LOG_FILE=database-build.log ++ date +%s.%N
• start_time=1662796473.356181557 ++ dirname /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/build_db.sh
• script_dir=/home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec
• DATABASE_DIR=/data_agro/DB_microbe
• FIND_OPTS=-L
• [[ '' == '' ]]
• JELLYFISH_BIN=/home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish
• NCBI_SERVER=ftp.ncbi.nih.gov
• FTP_SERVER=ftp://ftp.ncbi.nih.gov
• '[' '!' -d /data_agro/DB_microbe ']'
• cd /data_agro/DB_microbe
• MEMFLAG=
• '[' -z 1 ']'
• echo 'Kraken build set to minimize RAM usage.' Kraken build set to minimize RAM usage.
• '[' 0 == 1 ']'
• LIBRARY_DIR=library/
• [[ '' != '' ]]
• TAXONOMY_DIR=taxonomy/
• [[ '' != '' ]]
• '[' '!' -s library-files.txt ']' ++ cat library-files.txt ++ wc -l
• N_FILES=1
• [[ 1 -eq 0 ]]
• echo 'Found 1 sequence files (.{fna,fa,ffn,fasta,fsa}) in the library directory.' Found 1 sequence files (.{fna,fa,ffn,fasta,fsa}) in the library directory.
• '[' -e database.jdb ']'
• '[' -e database0.kdb ']'
• echo 'Creating k-mer set (step 1 of 6)...' Creating k-mer set (step 1 of 6)... ++ date +%s.%N
• start_time1=1662796473.400182088
• echo 'Using /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish' Using /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish
• [[ /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish != '' ]]
• '[' -z '' ']' ++ cat_library ++ count_unique -t 12 -k 31 ++ cat library-files.txt ++ tr '\n' '\0' ++ xargs -0 cat
• KRAKEN_HASH_SIZE=322671729911
• echo 'Hash size not specified, using '\''322671729911'\''' Hash size not specified, using '322671729911'
• exe eval /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish count -m 31 -s 322671729911 -C -t 12 -o database /dev/fd/63 ++ cat_library ++ date '+%Y-%m-%d %H:%M:%S' ++ cat library-files.txt ++ tr '\n' '\0' ++ xargs -0 cat
• echo -e '2022-09-10 15:32:56\teval' /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish count -m 31 -s 322671729911 -C -t 12 -o database /dev/fd/63
• eval /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish count -m 31 -s 322671729911 -C -t 12 -o database /dev/fd/63 ++ /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish count -m 31 -s 322671729911 -C -t 12 -o database /dev/fd/63 terminate called after throwing an instance of 'jellyfish::invertible_hash::ErrorAllocation' what(): Failed to allocate 2611340115968 bytes of memory /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/build_db.sh: line 46: 38099 Aborted (core dumped) /home/clusterusers/lborruso/.conda/envs/krakenuniq/libexec/jellyfish-install/bin/jellyfish count -m 31 -s 322671729911 -C -t 12 -o database /dev/fd/63 xargs: cat: terminated by signal 13

[StickHu]

Hi, have you solved the problem？