Closed mogrodnik closed 2 years ago
The problem with deadlocks in unrelated routines comes from repeat_step
not reduced globally before use. I guess that check_cfl_violation
routine also needs a fix. And the declaration of repeat_step
needs an elaborate comment with warning :)
I suppose that deadlocks in this case might have more than one cause and how repeat_step
is handled is just one way to make them happen. Well, anyway - the first time the deadlocks started to occur on our machines was after 4e9aed1cbf07a25ba5f9c1080aa6c802eaff68f9; i found it later, when I started to look over the network.
In 4e9aed1 call piernik_MPI_Bcast(repeat_step)
became conditional, which allowed having repeat_step
unreduced.
I just think that handling repeat_step
requires very defensive approach to avoid strange deadlocks in the future. Even if it will cost an extra synchronisaton call per timestep.
This fixes problems (hanging up) occuring on some larger simulations in which repetitive_steps are not allowed. Due to possible CFL violation 'repeat_step' flag was being triggered and in setups without repetitive steps, left unattended, this later causes the simulation to stall during an attempt to reduce chspeed (hdc:update_chspeed).