Layers should be testing for various supported activation functions (instead of just defaults).
Additionally, we need to make sure we are testing all layers equally extensively.
We did some work on this in 79d196482975ae8c0525130b6dd8c4866405d7ef, 27f933ceae5f79cbe354f2be9d7df595b5b06433 and e75be43a2cd83c4e81beed78c41695f2634e2dbc. Looks like we can close this for now.
Layers should be testing for various supported activation functions (instead of just defaults). Additionally, we need to make sure we are testing all layers equally extensively.