BlueBrain / nmodl

Code Generation Framework For NEURON MODeling Language
https://bluebrain.github.io/nmodl/
Apache License 2.0
53 stars 15 forks source link

Fix flakey test Crout vs. Eigen. #1469

Open 1uc opened 1 week ago

1uc commented 1 week ago

We're occasionally seeing:

Filters: "Scenario: Compare Crout solver with Eigen"
Randomness seeded to: 370099071
eigen_x_ColMajor vs crout_x_ColMajor (issue) / seed = 665289716

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
testcrout is a Catch2 v3.5.3 host application.
Run with -? for options

-------------------------------------------------------------------------------
Scenario: Compare Crout solver with Eigen
      Given: crout (double)
       Then: run tests & compare
-------------------------------------------------------------------------------
../test/unit/crout/crout.cpp:129
...............................................................................

../test/unit/crout/crout.cpp:130: FAILED:
  REQUIRE( test )
with expansion:
  false

eigen_timing [ms] : 32.2796
crout_timing [ms] : 34.7109
===============================================================================
test cases: 1 | 1 failed
assertions: 1 | 1 failed

This is likely due to a bug in the test, e.g. using random numbers. This needs to be made investigated and made to {fail,not fail} deterministically.