search

choderalab / yank

An open, extensible Python framework for GPU-accelerated alchemical free energy calculations.
http://getyank.org
MIT License
177 stars 70 forks source link

How to enable the parallel mode in two gpus server #1222

Open kingljy0818 opened 4 years ago

kingljy0818 commented 4 years ago

Dear all,

I have a two GPUS (1080Ti) server includes two CPUs, when I perform the test command:

build_mpirun_configfile "yank script --yaml=p-xylene-implicit.yaml"

I will get the error:

build_mpirun_configfile "yank script --yaml=p-xylene-implicit.yaml" Traceback (most recent call last): File "/home/anaconda3/bin/build_mpirun_configfile", line 6, in sys.exit(clusterutils.build_mpirun_configfile.main()) File "/home/anaconda3/lib/python3.7/site-packages/clusterutils-0.3.1-py3.7.egg/clusterutils/build_mpirun_configfile.py", line 248, in main File "/home/anaconda3/lib/python3.7/site-packages/clusterutils-0.3.1-py3.7.egg/clusterutils/build_mpirun_configfile.py", line 265, in figure_out_manager RuntimeError: Cannot determine job scheduler! Please ensure one of the following environment variables is set for your job: PBS: "PBS_GPUFILE" LSF: "LSB_HOSTS" SLURM: "SLURM_JOB_NODELIST"

Actually, I have tested the "p-xylene-implicit.yaml" script, which can normally run in one of the two GPUs. If I want to run this script using all the two GPUs and all the threads in the two CPUs, what should I do, I would need your help, many thanks.

Best

Jiyuan

hannahbrucemacdonald commented 4 years ago

Dear Jiyuan,

Thanks for your question! We are having issues with replica exchange between GPUs and we haven't got to the bottom of the issue yet ( Issue #1130 )! We would suggest running a simulation on a single GPU, and trying to parallelise in a different way --- i.e. running repeats or two different simulations on each GPU.

Please let me know if this answers your issue!

Thanks,

Hannah

kingljy0818 commented 4 years ago

Dear Hannah,

Thank you so much for your answer. In fact, the 2 1080ti GPU server was just tested whether Yank could use all GPUs to calculate the free energy of a large sets ligands. I am preparing to build a 10 GPUs and 2 CPUs server. Could I use Yank to calculate the absolute binding free energy of a 100 ligands sets to its target using all 10 GPU performance in the future? Would you give me a better suggestion? Many thanks.

Best

Jiyuan

andrrizzi commented 4 years ago

100 ligands sets to its target using all 10 GPU performance in the future?

Hi Jiyuan. Yes, the bug we're working on should not prevent you to parallelize YANK over ligands, as long as you keep 1 GPU per ligand. You don't have to use the build_mpirun_configfile if you are not working on a cluster. You can use mpirun as a normal MPI program.

kingljy0818 commented 4 years ago

Hi, andrrizzi. Thank you so much. My idea is the same as what you said, I have tried to calculate a ligand for each GPU.