norse / norse

Deep learning with spiking neural networks (SNNs) in PyTorch.
https://norse.github.io/norse/
GNU Lesser General Public License v3.0
664 stars 83 forks source link

Can Norse support no delay between input and output of spiking neurons? #370

Closed lucifer2859 closed 1 year ago

lucifer2859 commented 1 year ago

The input of spiking neurons will be accumulated into the membrane voltage in the next time step, which creates a delay of one time step between the input and output. As for deep SNNs, the delay between the input and output of the entire network will be higher.

Is there a delay free implementation method for pulse neurons in Norse?

The current input (input_tensor) is first integrated into the current (i_new) and then accumulated into the voltage (v_decayed)

Jegp commented 1 year ago

Good question @lucifer2859. Let me provide a quick answer and a somewhat more elaborate.

Quickly, yes there are delay free implementations for some neurons. But not for all. Luckily, it's straightforward to change existing neurons to not include the delay, simply by integrating the incoming current before calculating the neuron output (and not after, as it's done in some cases).

The slightly longer answer is that you're hitting a longer discussion we've had here #326 and here #325. The main problem, as I see it, is that Norse uses forward Euler integration, which means that we can efficiently encode our neuron dynamics, but slightly inaccurately compared to other solvers.

I'm happy to enter a discussion about how to do this more efficiently. I'm also equally happy to standardize the code-base so we have the same behavior everywhere. PR #326 implements this for quite some neuron models, and it would be natural to extend this.

Coming back to your question, can I ask about your use-case? Do you have a particular reason for requiring integrations without delays?

lucifer2859 commented 1 year ago

I want to verify if Norse can replicate the results of other frameworks (SpikingJelly and SNNTorch), and finally I found that the "lif_feed_forward_integral" function could be used to do so.

Jegp commented 1 year ago

Happy to hear that :-) Feel free to re-open if needed.