search

juliuskunze / jaxnet

Concise deep learning for JAX
Apache License 2.0
184 stars 14 forks source link

Feature Request: Add Mish Activation #19

Closed digantamisra98 closed 4 years ago

digantamisra98 commented 4 years ago

Mish is a new novel activation function proposed in this paper. It has shown promising results so far and has been adopted in several packages including:

All benchmarks, analysis and links to official package implementations can be found in this repository

It would be nice to have Mish as an option within the activation function group.

This is the comparison of Mish with other conventional activation functions in a SEResNet-50 for CIFAR-10: se50_1

juliuskunze commented 4 years ago

Hey Diganta, I am currently focusing on adding more fundamental features. I might reconsider this once the library is in a more advanced state, but will close the issue for now.

digantamisra98 commented 4 years ago

@JuliusKunze sure, no worries. Thanks. Feel free to give it a try and provide some feedback.