search

shankar1729 / jdftx

JDFTx: software for joint density functional theory
http://jdftx.org
82 stars 54 forks source link

Multiple GPUs #299

Closed ramadugu closed 1 year ago

ramadugu commented 1 year ago

Hello,

Please delete this issue if its not the right place/way to ask this question.

Can I run jdftx_gpu with multiple GPUs? We have two A100 cards on our compute node and we would like to take advantage of the two GPUs if jdftx can support it. A recent tutorial calculation's output is pasted here.

Thank you! Sai

JDFTx 1.7.0

Start date and time: Mon Sep 25 11:06:54 2023 Executable /projects/a9009/sbc538/tickets/jdftx-1.7.0/jdftx-1.7.0/jdftx/build/jdftx_gpu with command-line: -i si.in -o si.out Running on hosts (process indices): qgpu0206 (0) Divided in process groups (process indices): 0 (0) gpuInit: Found compatible cuda device 0 'NVIDIA A100-PCIE-40GB' gpuInit: Found compatible cuda device 1 'NVIDIA A100-PCIE-40GB' gpuInit: Selected device 0 Resource initialization completed at t[s]: 6.13 Run totals: 1 processes, 52 threads, 1 GPUs Memory pool size: 24576 MB (per process)

shankar1729 commented 1 year ago

Hi Sai,

Run with two MPI processes, and each one should pick up a different GPU. The Run Totals should then correspondingly report 2 GPUs.

So overall the limit is 1 GPU/process and 1 process/reduced k-pt, so you can scale JDFTx efficiently up to one GPU for every reduced k-pt (as reported in nStates).

Best, Shankar

ramadugu commented 1 year ago

Thank you. Running with MPI and two processes works perfectly with two GPUs.