Benchmark CI should fail w/ non-zero exit code if N/A output. Currently it just prints N/A in the table like this if something goes wrong
Case Pre Process Simulation Post Process
──────────────────────────────────────────────────────────────────
5eq_rk3_weno3_hllc 1.00x 0.99x 1.00x
viscous_weno5_sgb_mono 1.00x 1.01x 0.83x
ibm 0.50x N/A N/A
hypo_hll 1.00x 1.00x 1.00x
But it still "passes" CI with a checkmark. It should fail with a red stop sign!
Benchmark CI should fail w/ non-zero exit code if N/A output. Currently it just prints
N/A
in the table like this if something goes wrongBut it still "passes" CI with a checkmark. It should fail with a red stop sign!
Note that this N/A thing would probably come up much less often if https://github.com/MFlowCode/MFC/issues/394 and https://github.com/MFlowCode/MFC/issues/393 are fixed.
Also see this comment: https://github.com/MFlowCode/MFC/pull/413#issuecomment-2110742034