Added eigen versions of GELU, swish, mish, softplus, CDELU, and SELU

jatinchowdhury18 / RTNeural

Real-time neural network inferencing

BSD 3-Clause "New" or "Revised" License

607 stars 61 forks source link

Closed algoravioli closed 9 months ago

algoravioli commented 10 months ago

I added GELU, swish, mish, softplus, CDELU, SELU.

Made the mish activation function internally dependent on the softplus. Not sure if this was a good idea or not.

algoravioli commented 10 months ago

Thanks for the comments, I’ll rework this pull request and make it happen again. Shall we close this pull request?

jatinchowdhury18 commented 10 months ago

Sure thing!

Shall we close this pull request?

Up to you... if you'd like to keep it open and keep pushing code to this branch, that's fine with me!