WrightonLabCSU / DRAM

Distilled and Refined Annotation of Metabolism: A tool for the annotation and curation of function for microbial and viral genomes
GNU General Public License v3.0
239 stars 50 forks source link

Calling program has OMP_PROC_BIND set in its environment. Please unset OMP_PROC_BIND. #256

Open MonicaSteffi opened 1 year ago

MonicaSteffi commented 1 year ago

Dear All, I am trying to prepare databases for DRAM. I got the following error: DRAM-setup.py prepare_databases --output_dir DRAM_data /dss/dsshome1/0F/ra78zut/miniconda3/lib/python3.10/site-packages/mag_annotator/database_handler.py:123: UserWarning: Database does not exist at path None warnings.warn("Database does not exist at path %s" % description_loc) 2023-01-30 09:58:24,246 - Starting the process of downloading data 2023-01-30 09:58:24,246 - The kegg_loc argument was not used to specify a downloaded kegg file, and dram can not download it its self. So it is assumed that the user wants to set up DRAM without it 2023-01-30 09:58:24,246 - The gene_ko_link_loc argument was not used to specify a downloaded gene_ko_link file, and dram can not download it its self. So it is assumed that the user wants to set up DRAM without it 2023-01-30 09:58:24,247 - Database preparation started 2023-01-30 09:58:24,247 - Downloading kofam_hmm 2023-01-30 10:02:07,002 - Downloading kofam_ko_list 2023-01-30 10:02:09,498 - Downloading uniref 2023-01-31 10:58:31,363 - Downloading pfam 2023-01-31 11:08:06,871 - Downloading pfam_hmm 2023-01-31 11:08:07,261 - Downloading dbcan 2023-01-31 11:08:24,037 - Downloading dbcan_fam_activities 2023-01-31 11:08:24,037 - Downloading dbCAN family activities from : https://bcb.unl.edu/dbCAN2/download/Databases/V11/CAZyDB.08062022.fam-activities.txt 2023-01-31 11:08:24,741 - Downloading dbcan_subfam_ec 2023-01-31 11:08:24,742 - Downloading dbCAN sub-family encumber from : https://bcb.unl.edu/dbCAN2/download/Databases/V11/CAZyDB.08062022.fam.subfam.ec.txt 2023-01-31 11:08:25,865 - Downloading vogdb 2023-01-31 11:08:32,210 - Downloading vog_annotations 2023-01-31 11:08:32,390 - Downloading viral 2023-01-31 11:08:56,081 - Downloading peptidase 2023-01-31 11:09:06,384 - Downloading genome_summary_form 2023-01-31 11:09:06,774 - Downloading module_step_form 2023-01-31 11:09:07,038 - Downloading function_heatmap_form 2023-01-31 11:09:07,335 - Downloading amg_database 2023-01-31 11:09:07,545 - Downloading etc_module_database 2023-01-31 11:09:07,799 - All raw data files were downloaded successfully 2023-01-31 11:09:07,800 - Processing kofam_hmm 2023-01-31 11:14:39,238 - KOfam database processed 2023-01-31 11:14:39,530 - Moved kofam_hmm to final destination, configuration updated 2023-01-31 11:14:39,530 - Processing kofam_ko_list 2023-01-31 11:14:39,608 - KOfam ko list processed 2023-01-31 11:14:39,610 - Moved kofam_ko_list to final destination, configuration updated 2023-01-31 11:14:39,610 - Processing uniref 2023-01-31 11:34:24,433 - The subcommand ['mmseqs', 'createindex', 'DRAM_data/uniref90.20230131.mmsdb', 'DRAM_data/tmp', '--threads', '10'] experienced an error: Calling program has OMP_PROC_BIND set in its environment. Please unset OMP_PROC_BIND.

Traceback (most recent call last): File "/dss/dsshome1/0F/ra78zut/miniconda3/bin/DRAM-setup.py", line 184, in args.func(**args_dict) File "/dss/dsshome1/0F/ra78zut/miniconda3/lib/python3.10/site-packages/mag_annotator/database_processing.py", line 555, in prepare_databases processed_locs = process_functions[i](locs[i], output_dir, LOGGER, File "/dss/dsshome1/0F/ra78zut/miniconda3/lib/python3.10/site-packages/mag_annotator/database_processing.py", line 262, in process_uniref make_mmseqs_db(uniref_fasta_zipped, uniref_mmseqs_db, logger, create_index=True, threads=threads, verbose=verbose) File "/dss/dsshome1/0F/ra78zut/miniconda3/lib/python3.10/site-packages/mag_annotator/utils.py", line 98, in make_mmseqs_db run_process(['mmseqs', 'createindex', output_loc, tmp_dir, '--threads', str(threads)], logger, verbose=verbose) File "/dss/dsshome1/0F/ra78zut/miniconda3/lib/python3.10/site-packages/mag_annotator/utils.py", line 71, in run_process raise subprocess.SubprocessError(f"The subcommand {' '.join(command)} experienced an error, see the log for more info.") subprocess.SubprocessError: The subcommand mmseqs createindex DRAM_data/uniref90.20230131.mmsdb DRAM_data/tmp --threads 10 experienced an error, see the log for more info.

How do I rectify it? Any help would be appreciated. And also, if you have noticed, All raw data files were downloaded successfully. If I want to continue this preparing process, is there any way I can resume from this point? Downloading all the databases took forever for me.

Thank you in advance.

Regards Monica

nikolasbasler commented 1 year ago

To contribute an observation: I also got the first warning message at the very beginning (the one wih "Database does not exist at path None") when I prepared the databases recently. But then the rest of the preparation seems to finish fine. So I think, this might be unconnected to the errors you get further down.

I wanted to ask the devs at some point whether these warnings are a problem (that the database doesn't exist before it was downloaded is kind of a no-brainer), but I didn't have the time yet to test DRAM and see if it runs or not.

Using miniconda and DRAM version 1.4.6

rmFlynn commented 1 year ago

Thanks for informing us of this unique issue. First, the warning you experienced is totally normal. Database prep is expensive in time and memory, and some opt to replace existing databases. In this case, the use would want to be warned that they are building a database that did not previously exist. In your case it should not be a problem.

To solve your main issue, some other tool you are using has set the environment variable OMP_PROC_BIND I think you should do what mmseqs suggests in the error message and un set it. This is done with a bash command unset OMP_PROC_BIND You may want to do this right before you call DRAM in what ever script or tool or interface you use to do this.

I don't expect this to be a danger to other users on your machine but it may break another part of your work flow if OMP_PROC_BIND was being used by another tool other than DRAM/mmseqs. So you may want to look at its contents and save it somewhere before you unset the value, use echo $OMP_PROC_BIND to do that.

Good luck, let me know how it works for you.