nsuite validation fails—how to proceed?

arbor-sim / nsuite

A suite for benchmarking and validating NEURON, Nest and Arbor neuron network simulators/libraries.

Other

4 stars 9 forks source link

nsuite validation fails—how to proceed? #88

Open heplesser opened 5 years ago

heplesser commented 5 years ago

On one system I tested, nsuite validation results in

==  Running validation for arbor:
[FAIL] arbor rc-exp2syn-spike/default
[PASS] arbor rc-expsyn/default
[PASS] arbor rc-expsyn/highcurrent

The run.out file is

spike.abserr<0.03: fail

I am unsure how to proceed.

halfflat commented 4 years ago

Apologies for not getting back to you on this!

Our pass/fail criteria are too stringent to be useful in judging validity, but not informative enough to track improvement. The reported fail for arbor rc-exp2syn-spike/default is in fact not severe.

We really hope to address this Very Soon, by moving to quantitative reporting on deviations from reference solutions (issue #79(.