Add Depthwise 2D Convolution Layer

This layer is necessary to support various applications such as SV.

Depthwise convolution is a type of convolution in which each input channel is convolved with a different kernel (called a depthwise kernel).
Unlike a regular 2D convolution, depthwise convolution does not mix information across different input channels.

In Tensorflow They support like Below

tf.keras.layers.DepthwiseConv2D(
    kernel_size,
    strides=(1, 1),
    padding='valid',
    depth_multiplier=1,
    data_format=None,
    dilation_rate=(1, 1),
    activation=None,
    use_bias=True,
    depthwise_initializer='glorot_uniform',
    bias_initializer='zeros',
    depthwise_regularizer=None,
    bias_regularizer=None,
    activity_regularizer=None,
    depthwise_constraint=None,
    bias_constraint=None,
    **kwargs
)

In Pytorch They support like

torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)

groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

At groups=1, all inputs are convolved to all outputs.

At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels and producing half the output channels, and both subsequently concatenated.

At groups= in_channels, each input channel is convolved with its own set of filters (of size 
out_channels
in_channels
in_channels
out_channels

 ).

I would appreciate if you could provide your opinion on how nntrainer should support Depthwise Convolution.

We currently support Tensorflow Lite Export functionality. Would it be better to write in a way that is friendly to Tensorflow Lite?

In this case, we would need to recreate new layers.

Alternatively, would it be better to support it in a similar format to PyTorch?

this would require making changes to the logic of Conv2D since new parameters are involved and additional work may be required when exporting to Tensorflow Lite.

nnstreamer / nntrainer

Add Depthwise 2D Convolution Layer #2520