brainpy / BrainPy

Brain Dynamics Programming in Python
https://brainpy.readthedocs.io/
GNU General Public License v3.0
508 stars 92 forks source link

fix `brainpy.math.softplus` and `brainpy.dnn.SoftPlus` #581

Closed chaoming0625 closed 8 months ago

chaoming0625 commented 8 months ago

580

Dr-Chen-Xiaoyu commented 8 months ago

It seems that threshold needs to be divided by beta “return jnp.where( x > threshold/beta , ...” , based on the description of softplus "input X beta > threshold"

chaoming0625 commented 8 months ago

Exactly. Currently, the algorithm is:

jnp.where(x * beta > threshold, x, 1 / beta * jnp.logaddexp(beta * x, 0))

Thanks for the suggestion.