MFlowCode / MFC

Exascale simulation of multiphase/physics fluid dynamics
https://mflowcode.github.io
MIT License
132 stars 56 forks source link

Benchmark CI should fail w/ non-zero exit code if N/A output #415

Closed sbryngelson closed 1 month ago

sbryngelson commented 1 month ago

Benchmark CI should fail w/ non-zero exit code if N/A output. Currently it just prints N/A in the table like this if something goes wrong

  Case                     Pre Process   Simulation   Post Process  
 ────────────────────────────────────────────────────────────────── 
  5eq_rk3_weno3_hllc             1.00x        0.99x          1.00x  
  viscous_weno5_sgb_mono         1.00x        1.01x          0.83x  
  ibm                            0.50x          N/A            N/A  
  hypo_hll                       1.00x        1.00x          1.00x 

But it still "passes" CI with a checkmark. It should fail with a red stop sign!

Note that this N/A thing would probably come up much less often if https://github.com/MFlowCode/MFC/issues/394 and https://github.com/MFlowCode/MFC/issues/393 are fixed.

Also see this comment: https://github.com/MFlowCode/MFC/pull/413#issuecomment-2110742034

sbryngelson commented 1 month ago

Was closed by #423