Import ONNX model from pytorch - LDense only works over 2D tensors (LDense)

[FlorianThaler]

Hello.

I trained a model (using dense layers only) in stable-baselines, which I would like to include into an EDDL project. For this reason I imported the model in pytorch, which allows me to export the model in onnx format. When importing and building the onnx model in eddl I encounter the following error:

    ==================================================================
    ⚠️  LDense only works over 2D tensors (LDense) ⚠️
    ==================================================================

    terminate called after throwing an instance of 'std::runtime_error'
      what():  RuntimeError: LDense
    Aborted (core dumped)

For the sake of reproducibility I post the codes of a dummy example down below.

1. The definition of the stable-baselines model and the conversion to pytorch, and onnx (versions: onnx == 1.9.0, torch == 1.10.1, stable-baselines == 2.10.0):

     ```
     import torch as th
     import torch.nn as nn
   
     import gym
   
     from stable_baselines import PPO1
     from stable_baselines.common.policies import MlpPolicy
   
     class PyTorchMlp(nn.Module):  
         def __init__(self, n_inputs=4, n_actions=2):
             nn.Module.__init__(self)
   
             self.fc1 = nn.Linear(n_inputs, 64)
             self.fc2 = nn.Linear(64, 64)      
             self.fc3 = nn.Linear(64, n_actions)      
             self.activ_fn = nn.Tanh()
             self.out_activ = nn.Softmax(dim=0)
   
         def forward(self, x):
             x = self.activ_fn(self.fc1(x))
             x = self.activ_fn(self.fc2(x))
             x = self.out_activ(self.fc3(x))
             return x
   
     def copy_mlp_weights(baselines_model):
         torch_mlp = PyTorchMlp(n_inputs=4, n_actions=2)
         model_params = baselines_model.get_parameters()
   
         policy_keys = [key for key in model_params.keys() if "pi" in key]
         policy_params = [model_params[key] for key in policy_keys]
   
         for (th_key, pytorch_param), key, policy_param in zip(torch_mlp.named_parameters(), policy_keys, policy_params):
             param = th.from_numpy(policy_param)    
             #Copies parameters from baselines model to pytorch model
             print(th_key, key)
             print(pytorch_param.shape, param.shape, policy_param.shape)
             pytorch_param.data.copy_(param.data.clone().t())
   
         return torch_mlp
   
     env = gym.make("CartPole-v1")
     obs = env.reset()   
   
     model = PPO1(MlpPolicy, env, verbose = 0)
     model.learn(total_timesteps = 25)
   
     for key, value in model.get_parameters().items():
         print(key, value.shape)
   
     th_model = copy_mlp_weights(model)
   
     dummy_input = th.randn(4)
   
     th.onnx.export(th_model, dummy_input, 'test_0.onnx', opset_version = 9, export_params = True, verbose = True,\
                         input_names = ['input'], output_names = ['output'])
     ```

2. The C++ integration:

         #include <iostream>
   
         #include "eddl/apis/eddl.h"
         #include "eddl/serialization/onnx/eddl_onnx.h"
   
         using namespace eddl;
   
         int main(int argc, char **argv)
         {
   
             // import model
             Net* net = import_net_from_onnx_file("../data/test_0.onnx");
             // Net* net = import_net_from_onnx_file("../data/model.onnx");
             // build model
             build(net);
             // build(net,
             //     adam(0.01),                 // optimizer
             //     {"mean_squared_error"},     // losses
             //     {"categorical_accuracy"},   // metrics
             //     CS_CPU(),                   // CPU with maximum threads availables
             //     false);                     // no initialisation of weights
             Tensor* inpt = Tensor::randu({1, 4});
             summary(net);
   
             // auto outpt = predict(net, {inpt});
         }

As I found, a similar issue was already discussed in #328.

[salvacarrion]

Which version of the EDDL are you using? In theory, we support n-dimensional dense layers since the last version 1.1b, which is on par with the latest PyEDDL version (1.3.*)

[FlorianThaler]

I pulled the most recent version from the main branch.

The input the neural network takes is no multi-dimensional tensor - it is only an array with four floats. Is there Is there another way to import a neural network? Would it be feasible to import the weights of a neural network using the function eddl::set_parameters? Is there a detailed description of its functionality in the docu?

[salvacarrion]

If you are using the last version, the n-dense should work. We only support onnx for IO, although you could indeed try to load the weights "manually" using set_parameters.

Anyway, could you send me the onnx file to take a look and debug it?

[FlorianThaler]

onnx_model.zip

Sure - it would be great if you could have a look at it. It seems I can't add the file directly - so I compress it into a zip file. Hope it works ...

[jonandergomez]

Dear Florian,

Salva and myself have been working on the code of the EDDL to find the cause of the error.

The error is generated when importing from ONNX if the shape of the input layer(s) has only one dimension. The EDDL always assumes the first component of the shape is the batch_size. We corrected the problem with the C++ version with a provisional patch for checking whether the shape of input layers has only one dimension, then using it as the size of the input layer.

At least for the purpose of UC7 from the FRACTAL project, it runs OK now only in C++.

[FlorianThaler]

Great, this are good news.- thanks for the information. So, I just have to pull the most recent eddl version from github to get the patch, right?

BR, Florian

---

Von: Jon Ander Gómez Adrián @.*** Gesendet: Montag, 10. Oktober 2022 23:17 An: deephealthproject/eddl Cc: Thaler, Florian (0610xxx); Author Betreff: Re: [deephealthproject/eddl] Import ONNX model from pytorch - LDense only works over 2D tensors (LDense) (Issue #340)

Dear Florian,

Salva and myself have been working on the code of the EDDL to find the cause of the error.

The error is generated when importing from ONNX if the shape of the input layer(s) has only one dimension. The EDDL always assumes the first component of the shape is the batch_size. We corrected the problem with the C++ version with a provisional patch for checking whether the shape of input layers has only one dimension, then using it as the size of the input layer.

At least for the purpose of UC7 from the FRACTAL project, it runs OK now only in C++.

— Reply to this email directly, view it on GitHubhttps://github.com/deephealthproject/eddl/issues/340#issuecomment-1273829029, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5UXWGDBDPZZQQQM7ZCQJ3WCSBWDANCNFSM6AAAAAAQ245FBQ. You are receiving this because you authored the thread.Message ID: @.***>