Suggestion for making GTDB database

[Xinpeng021001]

Hi,

I followed the wiki to create the GTDB-database and I noticed in the final step there might be some errors:

find OPERA-MS-DB/ -type f -name '*.fna.gz' > OPERA-MS-DB/genomes_list.tx

It will give a empty list file and I guess it should be:

find -L OPERA-MS-DB/ -type f -name '*.fna.gz' > OPERA-MS-DB/genomes_list.txt

Otherwise when run the strain cluster step, it may give the error and fail at that part.

Best,

[jsgounot]

Hi Xinpeng,

thanks for letting me know.

Regards, JS

[Xinpeng021001]

Also forgot to mention: the threads function of the python program will give errors if using multiple threads(more than 1), I fixed manually and could send it later if needed.

Best Regards, Xinpeng

[jsgounot]

Oh, that's interesting, it works fine on my machine and others. I'm interested to see the error message (if you still have it) and the fix, thanks.

[Xinpeng021001]

I guess it might be my env/python version error if it works for you. Let me post it here:

python $WORK/final_course_project/glacier_algae/script/OPERA-MS/src_utils/make_operams_db_from_gtdb.py all_genomes.txt.gz all_taxonomy_r220.tsv.gz --outdir test --threads 16 Read taxonomy file Check taxonomic information Read genome file Check concordance Define genome size and seq numbers. Number of threads: 16 Traceback (most recent call last): File "/work/yinlab/xinpeng/final_course_project/glacier_algae/script/OPERA-MS/src_utils/make_operams_db_from_gtdb.py", line 148, in main() File "/work/yinlab/xinpeng/final_course_project/glacier_algae/script/OPERA-MS/src_utils/make_operams_db_from_gtdb.py", line 145, in main process(args) File "/work/yinlab/xinpeng/final_course_project/glacier_algae/script/OPERA-MS/src_utils/make_operams_db_from_gtdb.py", line 60, in process seqinfos = multi_threads_seqinfos(fnames) if args.threads > 1 else single_thread_seqinfos(fnames) File "/work/yinlab/xinpeng/final_course_project/glacier_algae/script/OPERA-MS/src_utils/make_operams_db_from_gtdb.py", line 109, in multi_threads_seqinfos with concurrent.futures.ProcessPoolExecutor(max_workers=args.threads) as executor: NameError: name 'args' is not defined

[Xinpeng021001]

The old code:

def multi_threads_seqinfos(fnames): seqinfos = {} with concurrent.futures.ProcessPoolExecutor(max_workers=args.threads) as executor: if USED_TQDM: iterator = tqdm.tqdm(executor.map(fasta_info, fnames), total=len(fnames)) else: iterator = executor.map(fasta_info, fnames)

    for seqres in iterator:
        seqinfos.update(seqres)

return seqinfos

and the fixed:

def multi_threads_seqinfos(fnames, threads): seqinfos = {} with concurrent.futures.ProcessPoolExecutor(max_workers=threads) as executor: if USED_TQDM: iterator = tqdm.tqdm(executor.map(fasta_info, fnames), total=len(fnames)) else: iterator = executor.map(fasta_info, fnames)

    for seqres in iterator:
        seqinfos.update(seqres)

return seqinfos

[Xinpeng021001]

by the way, you forget a "t" in the find command :)

find -L OPERA-MS-DB/ -type f -name '*.fna.gz' > OPERA-MS-DB/genomes_list.tx

find -L OPERA-MS-DB/ -type f -name '*.fna.gz' > OPERA-MS-DB/genomes_list.txt

[jsgounot]

This is weird, I should have caught this issue before. Thanks for letting me know.

[Xinpeng021001]

my pleasure :)