STEllAR-GROUP / octotiger

Astrophysics program simulating the evolution of star systems based on the fast multipole method on adaptive Octrees
http://octotiger.stellar-group.org/
Boost Software License 1.0
48 stars 18 forks source link

Octo-Tiger hangs on Fugaku again, but with larger node counts #455

Closed diehlpk closed 10 months ago

diehlpk commented 1 year ago

Octo-Tiger hangs here on Fugaku using more than 2000 nodes.

This content is in the stdcout file

SCF option file "scf.init" not found - using defaults
Initialize executors and masks...
Kokkos already initialized
Using Kokkos serial executors for multipole FMM kernels...
Using Kokkos serial executors for monopole FMM kernels...
Number of tasks per KOKKOS hydro kernel: 1
Initializing cell_geometry 3 8 14
Stencils initialized!
Initialize problem...
Compiled with max nf -DOCTOTIGER_WITH_MAX_NUMBER_FIELDS=15
Problem initialized!
Starting to compute ilist...
ilist initialized!
Normalized constants 222
1.000000e+00 1.000000e+00 3.871264e+03 1.000000e+00
A = 8.995508e+29 | B = 1.962022e+06 | G = 1.000000e+00 | kb = 2.069149e-09 | c = 1.160576e+14 | mh = 1.673300e-24 | sigma = 3.289883e+06 | h = 2.565129e-23

This content is in the stdcerr file

Start octotiger
Start processing options
Finished processing options
Finished executor init and read options
Finished initializing options
Finished computing factors
Finished static_init
Finished normalizing
Finished initialization
Finished init
Found root
Loading from X.0.silo ...
diehlpk commented 11 months ago

With the MPI update, Octo-Tiger hangs now here

Starting main...
Registering functions ...
Starting hpx init ...
Initializing Kokkos host executors...
Initializing Kokkos device executors...
CPPuddle config: Using 1 devices!
CPPuddle config: Using 128 internal pool instances!
Finished executor init and read options
Finished initializing options
Finished computing factors
Finished static_init
Finished normalizing
Finished initialization