davisking / dlib

A toolkit for making real world machine learning and data analysis applications in C++
http://dlib.net
Boost Software License 1.0
13.57k stars 3.38k forks source link

Add transpose_ layer for 2D matrix transposition in 4D Dlib tensors #3013

Closed Cydral closed 1 month ago

Cydral commented 2 months ago

This PR adds a new transpose_ layer to the Dlib deep learning framework.

The transpose_ layer performs 2D matrix transposition on each k-plane of a 4D tensor:

template <typename SUBNET>
using transpose = add_layer<transpose_, SUBNET>;

Usage example:

using net_type = loss_multiclass_log<
                    fc<number_of_classes,
                    transpose<
                    conv<16,5,5,2,2,input<matrix<float>>>
                    >>>;

This layer is useful for operations that require transposed feature maps, such as certain types of attention mechanisms or specialized convolution patterns.

davisking commented 1 month ago

Nice, this is great. Thanks for the PR :)