search

yezhengSTAT / scVI-3D

GNU General Public License v3.0
5 stars 2 forks source link

Terminating: fork() called from a process already using GNU OpenMP, this is unsafe. #9

Open Accompany0313 opened 3 months ago

Accompany0313 commented 3 months ago

Hi Ye，

Nice work!

I got an error: " /MNT/second_drive/user_data ygc/local/lib/python3.8 / site - packages/sklearn/utils/deprecation py: 86: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0. warnings.warn(msg, category=FutureWarning) INFO Using batches from adata.obs["batch"] INFO No label_key inputted, assuming all cells have same label INFO Using data from adata.X Terminating: fork() called from a process already using GNU OpenMP, this is unsafe. / MNT/second_drive/user_data ygc/local/lib/python3.8 / site - packages/sklearn/utils/deprecation py: 86: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0. warnings.warn(msg, category=FutureWarning) INFO Using batches from adata.obs["batch"] INFO No label_key inputted, assuming all cells have same label INFO Using data from adata.X Terminating: fork() called from a process already using GNU OpenMP, this is unsafe."

I have no idea what could cause the error. The command I use is: python scripts/scVI-3D.py -b 10 -c "whole" -r 1000000 -i "demoData" -o "results" -cs "supplementaryData/demoData_summary.txt" -g "supplementaryData/hg19.chrom.sizes" -br -n 100 -gpu -p 10 -pca 50 -up -tp -v

Could you please help me with it? Thank you so much!

yezhengSTAT commented 3 months ago

Hello, It seems that the newer version of sklearn package that you probably is using has deprecated setup_anndata function. I think we developed the package using 0.24.2 version of scikit-learn (for other python module, the version we used can be found at https://github.com/yezhengSTAT/scVI-3D#1-preparation). Sorry for the trouble as we did not expect the new dependent tool is not compatible with its old version.

Thanks, Ye

yezhengSTAT commented 3 months ago

The version compatibility issue indeed needs some further adjustment work, but I guess that will only give us warning messages. Terminating errors related to fork() and GNU OpenMP is the critical error stopping your process. The error message "Terminating: fork() called from a process already using GNU OpenMP, this is unsafe" suggests that your script or one of the libraries it uses tries to use multiprocessing in a way that's not safe or compatible with OpenMP (Open Multi-Processing), which is used for parallel programming in C/C++ and Fortran. This error can occur when a process that's already running in a parallelized manner (using OpenMP) attempts to spawn new processes (via fork() in Python). This is often unsafe because it can lead to deadlocks or other concurrency issues. Maybe try to use one thread by setting OMP_NUM_THREADS=1?

Accompany0313 commented 3 months ago

Thank you very much for your advice. I tried to set OMP_NUM_THREADS=1 first, but the same problem still occurred. I tried again to set the number of cpus to 1, and this time the problem was resolved. However, the running time has been extended from 15 minutes to more than two hours. Finally, I would like to express my sincere thanks for your reply.

The command I use is: python scripts/scVI-3D.py -b 10 -c "whole" -r 1000000 -i "demoData" -o "results" -cs "supplementaryData/demoData_summary.txt" -g "supplementaryData/hg19.chrom.sizes" -br -n 100 -gpu -p 1 -pca 50 -up -tp -v