search

Sergio0694 / NeuralNetwork.NET

A TensorFlow-inspired neural network library built from scratch in C# 7.3 for .NET Standard 2.0, with GPU support through cuDNN
GNU General Public License v3.0
547 stars 86 forks source link

Feature Request: Add Mish activation #93

Closed digantamisra98 closed 3 years ago

digantamisra98 commented 4 years ago

Mish is a new novel activation function proposed in this paper. It has shown promising results so far and has been adopted in several packages including:

All benchmarks, analysis and links to official package implementations can be found in this repository

It would be nice to have Mish as an option within the activation function group.

This is the comparison of Mish with other conventional activation functions in a SEResNet-50 for CIFAR-10: (Better accuracy and faster than GELU) se50_1

JamesLear92 commented 4 years ago

I've just raised a pull request to implement this:

94

digantamisra98 commented 4 years ago

@JamesLear92 Thanks for the consideration

JamesLear92 commented 4 years ago

@digantamisra98 I've now got it working on cuda so the pull request is ready for merge

digantamisra98 commented 4 years ago

@JamesLear92 awesome. I'll look forward to giving it a try once merged. Thanks!

JamesLear92 commented 4 years ago

@Sergio0694 Hi, are you happy to merge the my pull request?