ReduceSum op: AttributeError: module 'numpy' has no attribute 'astype'

[medvedev]

The ReduceSum operation in an input ONNX file results into the AttributeError: module 'numpy' has no attribute 'astype' in rknn.build(...)

Code to reproduce:

import torch
import torch.nn as nn

# Create a model and export it to ONNX
path_to_onnx = 'sum-demo.onnx'

class Demo(nn.Module):

    def forward(self, x):
        return torch.sum(x)

torch.onnx.export(Demo(),
                  (torch.randn(8),),
                  path_to_onnx,
                  opset_version=17,
                  input_names=['x',],
                  output_names=['output',],)

# Convert ONNX to RKNN

from rknn.api import RKNN

rknn = RKNN(verbose=True, verbose_file='./convert.log')

rknn.config(
    mean_values=[[0]],
    std_values=[[0]],
    quant_img_RGB2BGR=False,
    target_platform='rk3588')

rknn.load_onnx(model=path_to_onnx)

# Dataset contains path to only one .npy file, containing 1D numpy array of 8 floats
# Example code to generate such npy file:
# import numpy as np
# path_to_npy = "1d-array.npy"
# np.save(path_to_npy, np.random.rand(8))
rknn.build(do_quantization=True, dataset='<path to dataset>')

Output:

...
D base_optimize ...
D base_optimize done.
D 
D fold_constant ...
/home/<username>/miniconda3/envs/toolkit2/lib/python3.8/site-packages/rknn/api/rknn.py:204: RuntimeWarning: divide by zero encountered in divide
  return self.rknn_base.build(do_quantization=do_quantization, dataset=dataset, expand_batch_size=rknn_batch_size)
D fold_constant done.
D 
D correct_ops ...
D correct_ops done.
D 
D fuse_ops ...
W build: Show op fuse match nodes:
Ruler: unsqueeze_to_4d_reducesum
Subgraph:
Op type: ReduceSum
Op name: /ReduceSum
  Input:
    x : [8]
  Output:
    output : []
  Attribute:
    keepdims : 0

E build: Traceback (most recent call last):
  File "rknn/api/rknn_log.py", line 309, in rknn.api.rknn_log.error_catch_decorator.error_catch_wrapper
  File "rknn/api/rknn_base.py", line 1901, in rknn.api.rknn_base.RKNNBase.build
  File "rknn/api/graph_optimizer.py", line 2169, in rknn.api.graph_optimizer.GraphOptimizer.fuse_ops
  File "rknn/api/rules/dims.py", line 807, in rknn.api.rules.dims._p_unsqueeze_to_4d
  File "rknn/api/rules/utils.py", line 172, in rknn.api.rules.utils.Graph.op
  File "/home/<username>/.local/lib/python3.8/site-packages/numpy/__init__.py", line 320, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'astype'

Environment: OS architecture: Linux x86_64 Python: 3.8.19 Toolkit version: 2.0.0b17+a7d83749

[medvedev]

I really appreciate the fact that the problem is partially solved in build 20, but now with the same code and python 3.10.14 I face a new error:

...
D fuse_ops done.
D 
D sparse_weight ...
E build: Traceback (most recent call last):
  File "rknn/api/rknn_log.py", line 309, in rknn.api.rknn_log.error_catch_decorator.error_catch_wrapper
  File "rknn/api/rknn_base.py", line 1906, in rknn.api.rknn_base.RKNNBase.build
  File "rknn/api/graph_optimizer.py", line 2402, in rknn.api.graph_optimizer.GraphOptimizer.sparse_weight
  File "rknn/api/sparse_weight.py", line 528, in rknn.api.sparse_weight.sparse_weight
  File "rknn/api/sparse_weight.py", line 55, in rknn.api.sparse_weight._range_input
  File "/home/<username>/miniconda3/envs/<env_name>/lib/python3.10/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position 0: invalid start byte

[medvedev]

And another error for another attempt to get ReduceSum converted to RKNN: this time, use Pytorch's dynamo_export. Python: 3.10.14 rknn toolkit: 2.0.0b20 onnxruntime package versions: tried 1.16.0 and 1.18.1, same result

Code:

import torch
import torch.nn as nn

# Create a model and export it to ONNX
path_to_onnx = 'sum-demo.onnx'

class Demo(nn.Module):
    def forward(self, x):
        return torch.sum(x)

# Convert ONNX to RKNN
onnx_program = torch.onnx.dynamo_export(Demo(), torch.randn(8),)
onnx_program.save(path_to_onnx)

from rknn.api import RKNN

rknn = RKNN(verbose=True, verbose_file='./convert.log')

rknn.config(
    mean_values=[[0]],
    std_values=[[0]],
    quant_img_RGB2BGR=False,
    target_platform='rk3588')

rknn.load_onnx(model=path_to_onnx)

# Dataset contains path to only one .npy file, containing 1D numpy array of 8 floats
# Example code to generate such npy file:
import numpy as np
path_to_npy = "1d-array.npy"
np.save(path_to_npy, np.random.rand(8))
rknn.build(do_quantization=True, dataset=path_to_npy)

Error:

D fold_constant ...
E build: Traceback (most recent call last):
  File "rknn/api/rknn_log.py", line 309, in rknn.api.rknn_log.error_catch_decorator.error_catch_wrapper
  File "rknn/api/rknn_base.py", line 1885, in rknn.api.rknn_base.RKNNBase.build
  File "rknn/api/graph_optimizer.py", line 949, in rknn.api.graph_optimizer.GraphOptimizer.fold_constant
  File "rknn/api/session.py", line 34, in rknn.api.session.Session.__init__
  File "rknn/api/session.py", line 131, in rknn.api.session.Session.sess_build
  File "/home/<username>/miniconda3/envs/<env_name>/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/home/<username>/miniconda3/envs/<env_name>lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 462, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : This is an invalid model. In Node, ("_aten_sum_dim_none_0", _aten_sum_dim_none, "pkg.onnxscript.torch_lib", -1) : ("arg0": tensor(float),) -> ("sum_1": tensor(float),) , Error No opset import for domain 'pkg.onnxscript.torch_lib'