search

fangwei123456 / Spike-Element-Wise-ResNet

Deep Residual Learning in Spiking Neural Networks
Mozilla Public License 2.0
140 stars 21 forks source link

About grad function #9

Open Dignity-ghost opened 2 years ago

Dignity-ghost commented 2 years ago

Dear author, what kind of grad function do you used in SEW cext_neuron.MultiStepIFNode? Sigmoid?

fangwei123456 commented 2 years ago

Hi, I use ArcTan: https://spikingjelly.readthedocs.io/zh_CN/latest/spikingjelly.clock_driven.surrogate.html#atan-init-en

https://github.com/fangwei123456/spikingjelly/blob/2958519df84ad77c316c6e6fbfac96fb2e5f59a3/spikingjelly/cext/neuron.py#L864

Dignity-ghost commented 2 years ago

Hi, I use ArcTan: https://spikingjelly.readthedocs.io/zh_CN/latest/spikingjelly.clock_driven.surrogate.html#atan-init-en

https://github.com/fangwei123456/spikingjelly/blob/2958519df84ad77c316c6e6fbfac96fb2e5f59a3/spikingjelly/cext/neuron.py#L864

Thanks for replying. May I ask did you test ATan is more efficient than Sigmoid?

fangwei123456 commented 2 years ago

Does efficient mean higher accuracy? In my early experiments, ArcTan does have higher accuracy than Sigmoid. But it may depend on network structure. So, I suggest you can test different surrogates in https://spikingjelly.readthedocs.io/zh_CN/latest/spikingjelly.clock_driven.surrogate.html .

Dignity-ghost commented 2 years ago

Does efficient mean higher accuracy? In my early experiments, ArcTan does have higher accuracy than Sigmoid. But it may depend on network structure. So, I suggest you can test different surrogates in https://spikingjelly.readthedocs.io/zh_CN/latest/spikingjelly.clock_driven.surrogate.html .

Yes, I got it. Thank you very much.