sxs-collaboration / spectre

SpECTRE is a code for multi-scale, multi-physics problems in astrophysics and gravitational physics.
https://spectre-code.org
Other
154 stars 187 forks source link

Deadlock analysis only runs in the Exit phase #6010

Open wthrowe opened 1 month ago

wthrowe commented 1 month ago

This is documented, but it doesn't seem like desirable behavior. If the executable hangs in anything other than the last phase, run_deadlock_analysis_simple_actions is not called. A workaround is to remove all the phases after the one that locks up from the executable's default_phase_order.

knelli2 commented 1 month ago

So basically after every phase, all elements must report to the Main chare that they have successfully completed, otherwise deadlock analysis is run?

wthrowe commented 1 month ago

Hmm, yeah, I guess that would be required, which is a bit unpleasant.

I don't remember what the meanings of all the terminate-like flags are. Does each element know when the phase is over, independent of quiescence, or are all of the phase termination types potentially temporary? If they do, we could do the check at the end of the phase without needing an extra round trip message to Main.

knelli2 commented 1 month ago

No, each individual element doesn't know when the phase is over. They just know if they have completed or are waiting for another communication. Only Main knows when there's a phase change because it is the one that starts the next phase.