Implement cuDNN activation functions.

feiwang3311 / Lantern

BSD 3-Clause "New" or "Revised" License

167 stars 15 forks source link

Closed dan-zheng closed 5 years ago

dan-zheng commented 5 years ago

Add wrapper functions for cudnnActivationForward and cudnnActivationBackward.
Implement relu/tanh/sigmoid (original and gradient functions) using cuDNN, add tests.

TODO:

Support activations for non-4D tensors. This will likely require shape padding with "1" dimensions to work with the cuDNN API.