search

praisethemoon / cgraph

C computation graph, AutoGrad with OpenCL support [WIP]
Other
9 stars 1 forks source link

Add a numerical stable version of softmax #13

Open praisethemoon opened 5 years ago

praisethemoon commented 5 years ago

Add a numerical stable version of softmax

https://deepnotes.io/softmax-crossentropy http://www.nowozin.net/sebastian/blog/streaming-log-sum-exp-computation.html

praisethemoon commented 5 years ago

Another solution for sigmoid: https://timvieira.github.io/blog/post/2014/02/11/exp-normalize-trick/