search

hystrath / hyStrath

Hypersonic / Rarefied gas dynamics code developments (GPL-3.0)
https://hystrath.github.io/
GNU General Public License v3.0
219 stars 109 forks source link

minMaxTemperatureFieldOutOfRange not reporting accurately when running in parallel #155

Open croquemadame opened 4 months ago

croquemadame commented 4 months ago

Describe the bug When running in parallel, I am getting a warning similar to the one pasted below.

Attempt to use rho2ReactionThermo out of temperature range 263635 times during this iteration. Thigh: 40000 < 240000

The second line caught my attention since it is an exact multiple of the number of processors I am using, np, times Thigh, i.e, np*Thigh.

I looked in the code and it seems to me the problem lies in the following lines

https://github.com/hystrath/hyStrath/blob/984e3000a5f8d2e47cee555ec466d58ea9a6315d/src/thermophysicalModels/strath/strathReactionThermo/reactionThermo/rho2ReactionThermo/rho2ReactionThermo.H#L319

and

https://github.com/hystrath/hyStrath/blob/984e3000a5f8d2e47cee555ec466d58ea9a6315d/src/thermophysicalModels/strath/strathReactionThermo/reactionThermo/rho2ReactionThermo/rho2ReactionThermo.H#L320

To Reproduce Any case where the temperature limit is exceeded, running in parallel. In my case it is actually the lower bound that is exceeded, but I am receiving a warning about the upper bound. I believe this is happening because the check that is made

https://github.com/hystrath/hyStrath/blob/984e3000a5f8d2e47cee555ec466d58ea9a6315d/src/thermophysicalModels/strath/strathReactionThermo/reactionThermo/rho2ReactionThermo/rho2ReactionThermo.H#L328

returns false since the value stored in minMaxTemperatureFieldOutOfRange[0] is something close to 600 (6*100) instead of the lowest temperature in the domain, which I can see in my simulation results before the crash is 100 K.

Expected behavior The code should report the highest/lowest offending temperature in the domain. I believe the issue can be fixed by substituting sumOp<scalar>() by maxOp<scalar>() and sumOp<scalar>() by minOp<scalar>(), accordingly.

Environment

echo $WM_PROJECT_DIR
/opt/myapps/spack/0.20.1/opt/spack/linux-rocky8-ivybridge/gcc-7.5.0/openfoam-1706-xbli2le5cvcrkzi3l7wab3aruft2zjzj
echo $WM_PROJECT_USER_DIR
/home/juan/OpenFOAM/juan-v1706
echo $FOAM_USER_LIBBIN
/home/juan/OpenFOAM/juan-v1706/platforms/linux64GccDPInt32-spack/lib
echo $FOAM_USER_APPBIN
/home/juan/OpenFOAM/juan-v1706/platforms/linux64GccDPInt32-spack/bin

Additional context [ ] Is the bug specifically related to hyStrath? Yes [ ] Have you read the documentation on the hyStrath website? Yes [ ] Has it been covered in one or more Issues before? Not to the best of my knowledge

Add any other context about the problem here.

NB: You may be asked to attach a test case in a later stage.

croquemadame commented 4 months ago

I went ahead and made the suggested modification, and now I am getting an error message in accordance to the simulation results. I am attaching a patch.

0001-fixed-minMaxTemperatureFieldOutOfRange-parallel-issu.patch

Edit: Added proper patch file.