They treated ReLU and Swish in a unified manner and proposed the activation function ACtivationOrNot (ACON) as its general form. The activation function is composed of multiple learnable parameters, and they can freely change whether it is activated or not. They confirmed that this method improves the accuracy of object detection and image retrieval.
TL;DR
They treated ReLU and Swish in a unified manner and proposed the activation function ACtivationOrNot (ACON) as its general form. The activation function is composed of multiple learnable parameters, and they can freely change whether it is activated or not. They confirmed that this method improves the accuracy of object detection and image retrieval.
Why it matters:
Paper URL
https://arxiv.org/abs/2009.04759
Submission Dates(yyyy/mm/dd)
2019/09/10
Authors and institutions
Ningning Ma, Xiangyu Zhang, Jian Sun
Methods
Results
Comments