Run on GPU

[Eimantas10]

Hello,

I'm a beginner with Nbody simulations, but I cannot find the information of how to run the simulation on GPU. CPU, CPU with MPI works perfect. I have CUDA installed, so how can I run a simulation full on GPU ?

[kaiwu-astro]

Hi Eimantas,

NBODY6++GPU supports CUDA compiler version 8.0 only. You can check with nvcc --versions, and see if it shows 8.0 in the end. If not, maybe you need to install from https://developer.nvidia.com/cuda-80-ga2-download-archive

after that, you can do

git clone https://github.com/kaiwu-astro/Nbody6PPGPU-beijing
cd Nbody6PPGPU-beijing
./configure --with-par=b1m --enable-simd=sse --enable-mcmodel=large

Now you should see CUDA compiler followed by nvcc in the configure log, like

configure: --Compilers: configure: C++ compiler: g++ configure: Fortran compiler: gfortran configure: CUDA compiler: nvcc

that means NBODY6++GPU knows your CUDA compiler!

make clean ; make -j 

I also provides an test input file (1000 stars, 0.5 Myr) below so you can start quickly

mkdir test; 
cp -p ./build/nbody6++* ./test/; 
cd test; 
wget https://github.com/kaiwu-astro/garage/raw/main/Nbody6ppGPU-inputfile/kai_1k_no_dat-10/1k.inp; 

ulimit -s unlimited; 
export execname=`ls nbody6++*`; 
./$execname < 1k.inp

Cheers, Kai

[Eimantas10]

Kai,

Thank You for Your response.

I use CUDA 11.6 version, but configure log looks exactly as You described.

configure: ----------------------Configure Summary-------------------- configure: --Features: configure: Using MPI: yes configure: Using GPU: yes configure: CUDA version: cuda_11.6.r11.6/compiler.30794723_0 configure: Using SIMD: sse configure: mcmodel: large configure: OpenMP option: -fopenmp configure: Debug mode: no configure: HDF5 format: no configure: TT package: no configure: --Compilers: configure: C++ compiler: g++ configure: Fortran compiler: gfortran configure: CUDA compiler: nvcc

Therefore, I followed the steps that You have written with the test input (I just changed the number of stars to 10 000) and the code calculates everything without any errors. But I checked the CPU and GPU memory usage. CPU was at 80-90%, meanwhile GPU was only at ~200 Mb (although GPU that I use has 8 Gb of free memory). So my follow up question, is it normal that it uses so little memory of GPU ? For me it seems, that it only uses CPU. Is it possible to run calculations only on GPU, without using CPU ? Maybe I should try downgrade my CUDA version to 8.0.

Cheers, Eimantas

[kaiwu-astro]

Hi Eimantas,

When you run the simulation, can you see nbody process in nvidia-smi? If you can, then NBODY6++GPU works with your CUDA, and that's normal!

Quick answer: if you want the GPU to play the leading role, you need to have at least 50k particles, and hopefully more than 100k. To modify the number of stars, increase the first number in the 2nd line (it is now 1000) of 1k.inp, then run again (no need to recompile).

For simulations with less than 50k particles, wo suggest not to use GPU, otherwise it will slow down the simulation due to the communication cost between CPU and GPU.

Cheers, Kai

[Eimantas10]

Kai,

When you run the simulation, can you see nbody process in nvidia-smi?

Yes, the nbody process was in nvidia-smi window.

If you can, then NBODY6++GPU works with your CUDA, and that's normal!

Understood.

Quick answer: if you want the GPU to play the leading role, you need to have at least 50k particles, and hopefully more than 100k.

Okay, thank You for the clarification.

That would be all from me.

Cheers, Eimantas