In signal processing, if using ReLU, the second derivative is zero and higher order information is lost. However, it can be successfully learned by using the sin function as an activation function. Good results in image regression and signal processing where higher order information is important.
TL;DR
In signal processing, if using ReLU, the second derivative is zero and higher order information is lost. However, it can be successfully learned by using the sin function as an activation function. Good results in image regression and signal processing where higher order information is important.
Why it matters:
Paper URL
https://arxiv.org/abs/2006.09661
Submission Dates(yyyy/mm/dd)
Authors and institutions
Methods
Results
Comments