search

nengo / nengo-loihi

Run Nengo models on Intel's Loihi chip
https://www.nengo.ai/nengo-loihi/
Other
35 stars 12 forks source link

Include regression tests for weights=True versus False #224

Open arvoelke opened 5 years ago

arvoelke commented 5 years ago

As noted in https://github.com/nengo/nengo-loihi/pull/74#issuecomment-496270060 it would be useful to include some tests that compare the performance of weights=True versus weights=False in order to detect any regression in the performance of DecodeNeurons relative to full-weights, and to serve as a basic benchmark for experimenting with variants and future improvements.

This could be made part of a more general research task that involves determining which situations make more-or-less of a difference.

arvoelke commented 5 years ago

Using the trick in https://github.com/nengo/nengo-loihi/issues/230#issuecomment-501916092 here's a comparison of weights=True versus weights=False on a 6D Legendre Memory Unit (LMU):

weights=True lmu_with_monkeypatch

weights=False lmu_without_monkeypatch

With all-to-all weights we get a fairly stable history of the input. But with DecodeNeurons things blow up systematically (I've found some variations that look better, but they are qualitatively similar). This is running on the actual hardware on the master branch. I was seeing the same thing in my thesis (which also included the improvements in #132).