deephealthproject / eddl

European Distributed Deep Learning (EDDL) library. A general-purpose library initially developed to cover deep learning needs in healthcare use cases within the DeepHealth project.
https://deephealthproject.github.io/eddl/
MIT License
34 stars 10 forks source link

Import ONNX file with a different input channel dimension #332

Closed lauracanalini closed 2 years ago

lauracanalini commented 2 years ago

Hi, I am trying to import an onnx file with import_net_from_onnx_file with a specified input_shape. If I change the width and height dimensions everything is ok, but if I change the channel dimension it doesn't work anymore. For example:

#include "eddl/apis/eddl.h"
#include "eddl/serialization/onnx/eddl_onnx.h"

using namespace eddl;

int main(int argc, char** argv) {

    download_resnet101(false, { 1, 224, 224 });

    // or
    Net* imported_net = import_net_from_onnx_file("/path/to/resnet101.onnx", { 1, 224, 224 });
}

Output:

Downloading resnet101.onnx
resnet101.onnx x
resnet101.onnx                100%[=================================================>] 169,80M  7,75MB/s    in 18s
Import ONNX...
-------------------------------
class:         Tensor
ndim:          4
shape:         (64, 3, 7, 7)
strides:       (147, 49, 7, 1)
itemsize:      9408
contiguous:    1
order:         C
data pointer:  000001C79467EB20
is shared:     0
type:          float (4 bytes)
device:        CPU (code = 0)
-------------------------------
-------------------------------
class:         Tensor
ndim:          4
shape:         (64, 1, 7, 7)
strides:       (49, 49, 7, 1)
itemsize:      3136
contiguous:    1
order:         C
data pointer:  000001C793DD6F70
is shared:     0
type:          float (4 bytes)
device:        CPU (code = 0)
-------------------------------
==================================================================
ÔÜá´©Å  Tensors with different size (Tensor::copy) ÔÜá´©Å
==================================================================

Is there a way to change the input layer to provide an input with a different number of channels?

chavicoski commented 2 years ago

Hi,

The channels dimension can not be changed because the pretrained kernels of the Conv layers have (usually) the same depth as the input. You can modify the height and width because it is just a matter of sliding the kernel window, but the depth of the channels has to be the same.

For those cases, You can try to replicate the image two times to get the three-channel input or add zeros or something to fill the other channels.