search

nest / nest-simulator

The NEST simulator
http://www.nest-simulator.org
GNU General Public License v2.0
544 stars 370 forks source link

Extend accuracy testing for precise models #1493

Open heplesser opened 4 years ago

heplesser commented 4 years ago

Accuracy testing for precise models should be extended in two directions:

  1. Currently, test_iaf_ps_dc_accuracy.sli and test_iaf_ps_dc_t_accuracy.sli only test iaf_psc_alpha_ps and iaf_psc_delta_ps. They should also test iaf_psc_exp_ps, iaf_psc_exp_ps_lossless and iaf_psc_alpha_canon.
  2. These tests accuracy in the presence of DC current only, which is not a very relevant test case. Could one create tests with realistic spike input using arbitrary precision C++ or Python libraries to generate reference data?
heplesser commented 4 years ago

Concerning the testing, the attached slide shows the effect of different ordering of terms in the membrane potential update loop. Note that in this case we only have constant-current external input and the error can be determined by computing the value at arbitrary time directly from the analytical solution.

HEP_CSE11_Reproducibility (dragged).pdf

stinebuu commented 3 years ago

@heplesser I have started looking into this again.

When it comes to point 1: adding iaf_psc_exp_ps and iaf_psc_exp_ps_lossless to test_iaf_ps_dc_accuracy.sli requires the use of expm1 in the update loop of the models, like you point out in the slide as well. They are currently using exp, but this is an easy fix. The use of expm1 is consistent with iaf_psc_alpha_ps so that's ok. iaf_psc_alpha_canon does not need any updates and can be put into the test directly. I don't know why it is not already there, but the model is deprecated, so this might be why (it can already be removed, I will fix this in another PR).

But, when it comes to test_iaf_ps_dc_t_accuracy.sli I am having problems with both iaf_psc_exp_ps_lossless (with expm1) and iaf_psc_alpha_canon, the error is too big (see screenshot below for details on iaf_psc_exp_ps_lossless. I am not too familiar with the models, so I am not quite sure how this might be fixed. Do you have any pointers?

image

I have not started looking at point 2 yet.

github-actions[bot] commented 3 years ago

Issue automatically marked stale!

stinebuu commented 2 years ago

Part 1 fixed with #2314. I think someone else needs to take a look at part 2., as I am not that familiar with the models.