KeyError, (CenterNet Pytorch)

[Hikaru-Furuta]

issue

I've tried to load an onnx-model that I had created using a module, "torch.onnx.export", from a ".pth"(Pytorch) model. The following codes are for these implimentaitions. Some nodes in my onnx-file couldn't be accepted by 'ng.from_onnx'. For example, 'ConvTranspose' is not supported in NNgen. As is written on NNgen's Git, I interpreted this as that existing models should be loaded from torchvision instead of creating onnx-file myself. Is there any possible ways to use my own onnx-file?

Converting a pytorch-file into onnx-file

from lib.models.networks.dlav0 import get_pose_net
from lib.models.model import load_model
import torch.onnx as onnx
import torch
from types import MethodType

def forward(self, x):
    x = self.base(x)
    x = self.dla_up(x[self.first_level:])
    ret = []
    for head in self.heads:
        ret.append(self.__getattr__(head)(x))
    return ret

input = torch.zeros([1, 3, 512, 512])
net = get_pose_net(34, {'hm':2, 'reg':2, 'wh':2})
net.forward = MethodType(forward, net)

load_model(net, '../models/ctdet_coco_dla_2x.pth')
onnx.export(net, input, '../models/ctdet_coco_dla_2x_2.onnx', verbose=True)

#Refered to 'https://github.com/CaoWGG/TensorRT-CenterNet/issues/1'
#Model, 'ctdet_coco_dla_2x.pth', downloaded from 'https://drive.google.com/open?id=1pl_-ael8wERdUREEnaIfqOV_VF2bEVRT'
#Centernet-Model description : https://github.com/xingyizhou/CenterNet

Loading my own onnx-file. (Getting_started.py )

from __future__ import absolute_import
from __future__ import print_function

import sys
import os

import nngen as ng
import torch
import torchvision

# data types
act_dtype = ng.int16
weight_dtype = ng.int16
bias_dtype = ng.int16
scale_dtype = ng.int16

dtypes = {}
(outputs, placeholders, variables,
 constants, operators) = ng.from_onnx('ctdet_coco_dla_2x.onnx',
                                      value_dtypes=dtypes,
                                      default_placeholder_dtype=act_dtype,
                                      default_variable_dtype=weight_dtype,
                                      default_constant_dtype=weight_dtype,
                                      default_operator_dtype=act_dtype,
                                      default_scale_dtype=scale_dtype,
                                      default_bias_dtype=bias_dtype,
                                      disable_fusion=disable_fusion)

if act_dtype.width > 8:
    act_scale_factor = 128
else:
    act_scale_factor = int(round(2 ** (act_dtype.width - 1) * 0.5))

input_scale_factors = {'act': act_scale_factor}
input_means = {'act': imagenet_mean * act_scale_factor}
input_stds = {'act': imagenet_std * act_scale_factor}

ng.quantize(outputs, input_scale_factors, input_means, input_stds)

Loading my own onnx-file

@:~/git/nngen/nngen/onnx$ python Getting_started.py 
Traceback (most recent call last):
  File "Getting_started.py", line 19, in <module>
    constants, operators) = ng.from_onnx('ctdet_coco_dla_2x.onnx')#,
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 330, in from_onnx
    visitor.visit(name)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 148, in visit
    node_op = node_func(self, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/conv.py", line 21, in Conv
    src_obj = visitor.visit(src)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 148, in visit
    node_op = node_func(self, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/act_func.py", line 53, in Relu
    return _act_func(operator.relu, visitor, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/act_func.py", line 37, in _act_func
    src_op = conv.Conv(visitor, src_node, act_func=method)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/conv.py", line 21, in Conv
    src_obj = visitor.visit(src)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 148, in visit
    node_op = node_func(self, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/act_func.py", line 53, in Relu
    return _act_func(operator.relu, visitor, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/act_func.py", line 29, in _act_func
    act_func=method)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/batchnormalization.py", line 45, in BatchNormalization
    act_func=act_func)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/conv.py", line 21, in Conv
    src_obj = visitor.visit(src)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 148, in visit
    node_op = node_func(self, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/concat.py", line 17, in Concat
    src_obj = visitor.visit(src)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 148, in visit
    node_op = node_func(self, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/act_func.py", line 53, in Relu
    return _act_func(operator.relu, visitor, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/act_func.py", line 29, in _act_func
    act_func=method)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/batchnormalization.py", line 45, in BatchNormalization
    act_func=act_func)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/conv.py", line 21, in Conv
    src_obj = visitor.visit(src)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 148, in visit
    node_op = node_func(self, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/concat.py", line 17, in Concat
    src_obj = visitor.visit(src)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 148, in visit
    node_op = node_func(self, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/act_func.py", line 53, in Relu
    return _act_func(operator.relu, visitor, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/act_func.py", line 29, in _act_func
    act_func=method)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/batchnormalization.py", line 45, in BatchNormalization
    act_func=act_func)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/conv.py", line 21, in Conv
    src_obj = visitor.visit(src)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 148, in visit
    node_op = node_func(self, node)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/concat.py", line 17, in Concat
    src_obj = visitor.visit(src)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 147, in visit
    node_func = _get_func(node.op_type)
  File "/home/user/anaconda3/envs/CenterNet_ONNX/lib/python3.6/site-packages/nngen/onnx/__init__.py", line 76, in _get_func
    return func_map[op_type]
KeyError: 'ConvTranspose'

[Hikaru-Furuta]

FileNotFoundError: Download the YOLOv3 model using Pytorch, such as 'https://github.com/ultralytics/yolov3'. Then extract it, and rename it as 'yolov3'

Traceback (most recent call last): File "darknet_yolov3_tiny.py", line 413, in rslt = run(silent=False, verilog_filename='tmp.v') File "darknet_yolov3_tiny.py", line 77, in run model = models.Darknet(cfg_filename, img_size).to('cpu') AttributeError: module 'models' has no attribute 'Darknet'

[Hikaru-Furuta]

asn1crypto = 0.24.0 cycler = 0.10.0 dataclasses = 0.8 future = 0.18.2 idna = 2.6 importlib-metadata = 3.1.1 keyring = 10.6.0 keyrings.alt = 3.0 kiwisolver = 1.3.1 MarkupSafe = 1.1.1

onnx = 1.8.0 opencv-python = 4.5.1.48 pip = 21.0 protobuf = 3.14.0 py = 1.9.0 pycrypto = 2.6.1 pygobject = 3.26.1 pyparsing = 2.4.7

python-apt = 1.6.5+ubuntu0.3 python-dateutil = 2.8.1

pyxdg = 0.25 SecretStorage = 2.3.1 torchvision = 0.8.1 unattended-upgrades = 0.1 veriloggen = 1.8.2

# matplotlib = 3.3.3 numpy = 1.19.4 packaging = 20.7

pyverilog = 1.2.1 setuptools = 39.0.1 six = 1.11.0 wheel = 0.30.0 pytest = 6.1.2

[RyusukeYamano]

FileNotFoundError: Download the YOLOv3 model using Pytorch, such as 'https://github.com/ultralytics/yolov3'. Then extract it, and rename it as 'yolov3'

Traceback (most recent call last): File "darknet_yolov3_tiny.py", line 413, in rslt = run(silent=False, verilog_filename='tmp.v') File "darknet_yolov3_tiny.py", line 77, in run model = models.Darknet(cfg_filename, img_size).to('cpu') AttributeError: module 'models' has no attribute 'Darknet'

It seems that the version of that repository 'ultralytics/yolov3' has been upgraded and is no longer supported, so checkout tags/v8 and give it a try.

[Hikaru-Furuta]

@RyusukeYamano Thanks for the replying I've already tried it but found it wouldn't work causing the same error

[ChristiaanBoe]

@Hikaru-Furuta

Have you already tried adding the map in the repository manually, that is how it worked for me. yolov3-map-20210215T163716Z-001.zip

[Hikaru-Furuta]

Thank you. I will try that

[ChristiaanBoe]

@Hikaru-Furuta

If it works out, I would appreciate you helping me with or sharing your version of the implementation on the FPGA as this is something I not yet managed to succeed in.