Add kEffNet 1.0 building block.

joaopauloschuler / neural-api

CAI NEURAL API - Pascal based deep learning neural network API optimized for AVX, AVX2 and AVX512 instruction sets plus OpenCL capable devices including AMD, Intel and NVIDIA.

GNU Lesser General Public License v2.1

356 stars 195 forks source link

Add kEffNet 1.0 building block. #63

Open joaopauloschuler opened 2 years ago

joaopauloschuler commented 2 years ago

Add kEffNet 1.0 building block described on the paper "Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet".

mikerabat commented 2 years ago

Just curious. Is the kEffNet building block capable of adding a 1D signal or is the convolution always 2D?

joaopauloschuler commented 2 years ago

Good question. This building block is described in these 2 papers:

Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet.
Grouped Pointwise Convolutions Reduce Parameters in Convolutional Neural Networks.

If your input has the 1x1xC shape (C is the number of channels), you can use this building block replacing dense layers.