search

GreenWaves-Technologies / gap_sdk

SDK for Greenwaves Technologies' GAP8 IoT Application Processor
https://greenwaves-technologies.com/en/gap8-the-internet-of-things-iot-application-processor/
Apache License 2.0
137 stars 75 forks source link

NNTOOL: Error when 'open *.onnx' #370

Closed ChenMathilda closed 1 year ago

ChenMathilda commented 1 year ago

Hi, I tried to use opencommand in nntool to open the onnx model generated by the parameters trained by pulp-dronet-v2, but I encountered the following error:

(NNT) open model_gapflow.onnx 
open - opening graph file model_gapflow.onnx load_quantization = False
onnx - unable to determine batch dimension. if the graph fails to import properly set it to 1 or a variable.
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk3.9/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk3.9/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk3.9/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/interpreter/commands/open.py", line 118, in do_open
    self.__open_graph(args)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/interpreter/commands/open.py", line 92, in __open_graph
    G = create_graph(graph_file, opts=opts)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/importer.py", line 52, in create_graph
    graph = importer.create_graph(filename, opts)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/onnx/onnx.py", line 54, in create_graph
    G = self._import_onnx_model(G, model.graph, opset_import, opts)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/onnx/onnx.py", line 68, in _import_onnx_model
    self._import_nodes(G, graph, handlers, all_nodes, outputs, opts)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/onnx/onnx.py", line 169, in _import_nodes
    params = handler.handle(OnnxNode(node), all_nodes=all_nodes, vars_dict=vars_dict,
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/onnx/handlers/handler.py", line 63, in handle
    raise BackendIsNotSupposedToImplementIt(
onnx.backend.test.runner.BackendIsNotSupposedToImplementIt: Constant version 13 is not implemented.
EXCEPTION of type 'BackendIsNotSupposedToImplementIt' occurred with message: 'Constant version 13 is not implemented.'

Here is the network dronet_v2_gapflow Here is the .pth file pulp_dronet_v2_10.pth Here is the .onnx file model_gapflow.onnx

And my environment are listed: python: 3.8 gap_sdk: 3.9.1

I was able to run the open command successfully before, but I don't know if I made some changes to the gap_sdk environment that caused this problem. In addition, I noticed that there is a bn modulein .pth file, but there is no such module in the generated onnx model.This is different from the onnx model in the official pulp-dronet-v2 is as follow: :model_original.onnx image

This question has been bothering me for a long time. I look forward to your reply!

sousoux commented 1 year ago

Your SDK is very old. Please update it. ONNX Constant operator is supported in opset version 13 in the latest SDK.

(https://github.com/GreenWaves-Technologies/gap_sdk/blob/master/tools/nntool/nntool/importer/onnx/handlers/backend/constant.py)

ChenMathilda commented 1 year ago

``> Your SDK is very old. Please update it. ONNX Constant operator is supported in opset version 13 in the latest SDK.

(https://github.com/GreenWaves-Technologies/gap_sdk/blob/master/tools/nntool/nntool/importer/onnx/handlers/backend/constant.py)

Hi @sousoux, I changed gap_sdk version into 4.22.0, and can run 'open *.onnx' successfully! But when I tried to runmake clean all run platform=gvsoc under path /pulp-dronet-v2/gapflow, I met this error:

(gap_sdk)
# chennaiting @ seu-netsi in ~/pulp-dronet/pulp-dronet-v2/gapflow on git:main x [20:05:39]
$ make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels/norm_transpose.c /home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels/copy.c
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
sudo rm -f BUILD_MODEL_SQ8BIT/GenTile
sudo rm -f -rf BUILD_MODEL_SQ8BIT
sudo rm -f -rf nntool_output
**make: *** No rule to make target '/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/generators/nntool_extra_generators.c', needed by 'BUILD_MODEL_SQ8BIT/GenTile'.  Stop.**

According to the error report, I deleted 55 lines of code in pulp-dronet-v2/gapflow/common/model_decl.mk

image It do make some progresses, but still encountered error:

# chennaiting @ seu-netsi in ~/pulp/pulp-dronet/pulp-dronet-v2/gapflow on git:master x [19:08:43]
make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels/norm_transpose.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels/copy.c  
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
rm -f BUILD_MODEL_SQ8BIT/GenTile
rm -f -rf BUILD_MODEL_SQ8BIT
rm -f -rf nntool_output 
mkdir BUILD_MODEL_SQ8BIT
cp nntool_input/models_onnx/model_original.onnx BUILD_MODEL_SQ8BIT/network.onnx
echo "GENERATING NNTOOL STATE FILE"
GENERATING NNTOOL STATE FILE
echo BUILD_MODEL_SQ8BIT
BUILD_MODEL_SQ8BIT
nntool -s nntool_input/nntool_scripts/nntool_script_deployment BUILD_MODEL_SQ8BIT/network.onnx -q
settings - set log level to INFO
log_level - was: 'INFO'
now: 'INFO'
open - opening graph file BUILD_MODEL_SQ8BIT/network.onnx load_quantization = True
__init__ - unable to determine batch dimension. if the graph fails to import properly set it to 1 or a variable.
nngraph - update graph dimensions
nngraph - update graph dimensions
debug - was: False
now: True
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
nngraph - update graph dimensions
move_node_up - Node _sig_Sigmoid cannot be moved
fuse_gap_convs - fusing nodes _layer1_Conv,_pool_MaxPool into _layer1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_conv1_Conv,_resBlock1_relu1_Clip into _resBlock1_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_conv2_Conv,_resBlock1_relu2_Clip into _resBlock1_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_bypass_Conv,_resBlock1_relu3_Clip into _resBlock1_bypass_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_conv1_Conv,_resBlock2_relu1_Clip into _resBlock2_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_conv2_Conv,_resBlock2_relu2_Clip into _resBlock2_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_bypass_Conv,_resBlock2_relu3_Clip into _resBlock2_bypass_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_conv1_Conv,_resBlock3_relu1_Clip into _resBlock3_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_conv2_Conv,_resBlock3_relu2_Clip into _resBlock3_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_bypass_Conv,_resBlock3_relu3_Clip into _resBlock3_bypass_Conv_fusion
matcher - ++ fusion fuse_gap_convs modified graph
slice_to_split - replaced slice nodes _Gather,_Gather_1 with split node _Gather_split
matcher - ++ fusion slice_to_split modified graph
insert_copies - inserting copy between _Gather_reshape0:0 and output_1:0
matcher - ++ fusion insert_copies modified graph
remove_reshapes_before_linear - removing unnecessary reshape before linear _Flatten
matcher - ++ fusion remove_reshapes_before_linear modified graph
fuse_op_activation - fusing nodes _resBlock3_Add,_relu_Clip
matcher - ++ fusion fuse_op_activation_scale8 modified graph
move_node_up - Node _sig_Sigmoid cannot be moved
matcher - ++ fusion scaled_match_group modified graph
move_node_up - Node _sig_Sigmoid cannot be moved
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
input_norm_func - was: ''
now: 'x:x/255'
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/aquant.py", line 99, in do_aquant
    data = [import_data(input_file, **input_args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/aquant.py", line 99, in <listcomp>
    data = [import_data(input_file, **input_args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 171, in import_data
    return import_image_data(filename, **kwargs)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 134, in import_image_data
    return postprocess(img_in, height, width, channels, **kwargs)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 86, in postprocess
    img_in = np.array(img_in, dtype=np.float)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
EXCEPTION of type 'AttributeError' occurred with message: 'module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations'
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/qshow.py", line 41, in do_qshow
    self._check_quantized()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 271, in _check_quantized
    raise GraphNotReadyException(CHECK_QUANTIZED_ERROR)
nntool.interpreter.nntool_shell_base.GraphNotReadyException: 
The opened graph must be quantized to use this command. Run the aquant command.

EXCEPTION of type 'GraphNotReadyException' occurred with message: '
The opened graph must be quantized to use this command. Run the aquant command.
'
nngraph - update graph dimensions
inserted image formatter after node input_1 withformat bw8 and normalization shift_int8
l3_ram_ext_managed - was: False
now: True
default_input_exec_location - was: 'AT_MEM_L2'
now: 'AT_MEM_L3_HRAM'
graph_produce_node_names - was: False
now: True
graph_reorder_constant_in - was: True
now: True
graph_produce_operinfos - was: False
now: True
graph_monitor_cycles - was: False
now: True
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/save_state.py", line 49, in do_save_state
    self._check_quantized()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 271, in _check_quantized
    raise GraphNotReadyException(CHECK_QUANTIZED_ERROR)
nntool.interpreter.nntool_shell_base.GraphNotReadyException: 
The opened graph must be quantized to use this command. Run the aquant command.

EXCEPTION of type 'GraphNotReadyException' occurred with message: '
The opened graph must be quantized to use this command. Run the aquant command.
'
echo "GENERATING AUTOTILER MODEL"
GENERATING AUTOTILER MODEL
nntool -g -M nntool_output -m networkModel.c -T nntool_output/tensors BUILD_MODEL_SQ8BIT/network.json
Traceback (most recent call last):
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/scripts/nntool", line 107, in <module>
    main()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/scripts/nntool", line 89, in main
    mod.generate_code(args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/generator.py", line 72, in generate_code
    nntool_shell.load_state_file(args.graph_file)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 374, in load_state_file
    with open(filepath) as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'BUILD_MODEL_SQ8BIT/network.json'
make: *** [common/model_rules.mk:49: nntool_output/networkModel.c] Error 1

According to debug, error reporting seems to be generated in the process of quantification: image It seems that the error is caused by the numpy version. My numpy==1.24.2. So I changed my numpy into numpy==1.23.0, and it works! But still met error:

echo "COMPILING AUTOTILER MODEL"
COMPILING AUTOTILER MODEL
gcc -g -o BUILD_MODEL_SQ8BIT/GenTile -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/generators -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators/CNN_Generator_Util.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/CNN_Generators_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.c nntool_output/networkModel.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler/LibTile.a  
In file included from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_BasicKernels_SQ8.h:21,
                 from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.h:21,
                 from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.c:21:
/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/../CNN_Libraries/CNN_Copy.h:23:10: fatal error: CNN_FloatType.h: No such file or directory
   23 | #include "CNN_FloatType.h"
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [common/model_rules.mk:56: BUILD_MODEL_SQ8BIT/GenTile] Error 1

I think this problem is caused by the large span of changing gap_sdkv3.9 to gap_sdkv4.22 (some file location may already changed). I wonder if there are any solutions? Or I can just modify the pulp-dronet's makefile by my own?

ChenMathilda commented 1 year ago

``> Your SDK is very old. Please update it. ONNX Constant operator is supported in opset version 13 in the latest SDK.

(https://github.com/GreenWaves-Technologies/gap_sdk/blob/master/tools/nntool/nntool/importer/onnx/handlers/backend/constant.py)

Hi @sousoux, I changed gap_sdk version into 4.22.0, and can run 'open *.onnx' successfully! But when I tried to runmake clean all run platform=gvsoc under path /pulp-dronet-v2/gapflow, I met this error:

(gap_sdk)
# chennaiting @ seu-netsi in ~/pulp-dronet/pulp-dronet-v2/gapflow on git:main x [20:05:39]
$ make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels/norm_transpose.c /home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels/copy.c
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
sudo rm -f BUILD_MODEL_SQ8BIT/GenTile
sudo rm -f -rf BUILD_MODEL_SQ8BIT
sudo rm -f -rf nntool_output
**make: *** No rule to make target '/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/generators/nntool_extra_generators.c', needed by 'BUILD_MODEL_SQ8BIT/GenTile'.  Stop.**

According to the error report, I deleted 55 lines of code in pulp-dronet-v2/gapflow/common/model_decl.mk

image It do make some progresses, but still encountered error:

# chennaiting @ seu-netsi in ~/pulp/pulp-dronet/pulp-dronet-v2/gapflow on git:master x [19:08:43]
make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels/norm_transpose.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels/copy.c  
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
rm -f BUILD_MODEL_SQ8BIT/GenTile
rm -f -rf BUILD_MODEL_SQ8BIT
rm -f -rf nntool_output 
mkdir BUILD_MODEL_SQ8BIT
cp nntool_input/models_onnx/model_original.onnx BUILD_MODEL_SQ8BIT/network.onnx
echo "GENERATING NNTOOL STATE FILE"
GENERATING NNTOOL STATE FILE
echo BUILD_MODEL_SQ8BIT
BUILD_MODEL_SQ8BIT
nntool -s nntool_input/nntool_scripts/nntool_script_deployment BUILD_MODEL_SQ8BIT/network.onnx -q
settings - set log level to INFO
log_level - was: 'INFO'
now: 'INFO'
open - opening graph file BUILD_MODEL_SQ8BIT/network.onnx load_quantization = True
__init__ - unable to determine batch dimension. if the graph fails to import properly set it to 1 or a variable.
nngraph - update graph dimensions
nngraph - update graph dimensions
debug - was: False
now: True
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
nngraph - update graph dimensions
move_node_up - Node _sig_Sigmoid cannot be moved
fuse_gap_convs - fusing nodes _layer1_Conv,_pool_MaxPool into _layer1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_conv1_Conv,_resBlock1_relu1_Clip into _resBlock1_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_conv2_Conv,_resBlock1_relu2_Clip into _resBlock1_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_bypass_Conv,_resBlock1_relu3_Clip into _resBlock1_bypass_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_conv1_Conv,_resBlock2_relu1_Clip into _resBlock2_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_conv2_Conv,_resBlock2_relu2_Clip into _resBlock2_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_bypass_Conv,_resBlock2_relu3_Clip into _resBlock2_bypass_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_conv1_Conv,_resBlock3_relu1_Clip into _resBlock3_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_conv2_Conv,_resBlock3_relu2_Clip into _resBlock3_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_bypass_Conv,_resBlock3_relu3_Clip into _resBlock3_bypass_Conv_fusion
matcher - ++ fusion fuse_gap_convs modified graph
slice_to_split - replaced slice nodes _Gather,_Gather_1 with split node _Gather_split
matcher - ++ fusion slice_to_split modified graph
insert_copies - inserting copy between _Gather_reshape0:0 and output_1:0
matcher - ++ fusion insert_copies modified graph
remove_reshapes_before_linear - removing unnecessary reshape before linear _Flatten
matcher - ++ fusion remove_reshapes_before_linear modified graph
fuse_op_activation - fusing nodes _resBlock3_Add,_relu_Clip
matcher - ++ fusion fuse_op_activation_scale8 modified graph
move_node_up - Node _sig_Sigmoid cannot be moved
matcher - ++ fusion scaled_match_group modified graph
move_node_up - Node _sig_Sigmoid cannot be moved
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
input_norm_func - was: ''
now: 'x:x/255'
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/aquant.py", line 99, in do_aquant
    data = [import_data(input_file, **input_args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/aquant.py", line 99, in <listcomp>
    data = [import_data(input_file, **input_args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 171, in import_data
    return import_image_data(filename, **kwargs)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 134, in import_image_data
    return postprocess(img_in, height, width, channels, **kwargs)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 86, in postprocess
    img_in = np.array(img_in, dtype=np.float)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
EXCEPTION of type 'AttributeError' occurred with message: 'module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations'
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/qshow.py", line 41, in do_qshow
    self._check_quantized()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 271, in _check_quantized
    raise GraphNotReadyException(CHECK_QUANTIZED_ERROR)
nntool.interpreter.nntool_shell_base.GraphNotReadyException: 
The opened graph must be quantized to use this command. Run the aquant command.

EXCEPTION of type 'GraphNotReadyException' occurred with message: '
The opened graph must be quantized to use this command. Run the aquant command.
'
nngraph - update graph dimensions
inserted image formatter after node input_1 withformat bw8 and normalization shift_int8
l3_ram_ext_managed - was: False
now: True
default_input_exec_location - was: 'AT_MEM_L2'
now: 'AT_MEM_L3_HRAM'
graph_produce_node_names - was: False
now: True
graph_reorder_constant_in - was: True
now: True
graph_produce_operinfos - was: False
now: True
graph_monitor_cycles - was: False
now: True
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/save_state.py", line 49, in do_save_state
    self._check_quantized()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 271, in _check_quantized
    raise GraphNotReadyException(CHECK_QUANTIZED_ERROR)
nntool.interpreter.nntool_shell_base.GraphNotReadyException: 
The opened graph must be quantized to use this command. Run the aquant command.

EXCEPTION of type 'GraphNotReadyException' occurred with message: '
The opened graph must be quantized to use this command. Run the aquant command.
'
echo "GENERATING AUTOTILER MODEL"
GENERATING AUTOTILER MODEL
nntool -g -M nntool_output -m networkModel.c -T nntool_output/tensors BUILD_MODEL_SQ8BIT/network.json
Traceback (most recent call last):
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/scripts/nntool", line 107, in <module>
    main()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/scripts/nntool", line 89, in main
    mod.generate_code(args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/generator.py", line 72, in generate_code
    nntool_shell.load_state_file(args.graph_file)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 374, in load_state_file
    with open(filepath) as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'BUILD_MODEL_SQ8BIT/network.json'
make: *** [common/model_rules.mk:49: nntool_output/networkModel.c] Error 1

According to debug, error reporting seems to be generated in the process of quantification: image It seems that the error is caused by the numpy version. My numpy==1.24.2. So I changed my numpy into numpy==1.23.0, and it works! But still met error:

echo "COMPILING AUTOTILER MODEL"
COMPILING AUTOTILER MODEL
gcc -g -o BUILD_MODEL_SQ8BIT/GenTile -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/generators -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators/CNN_Generator_Util.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/CNN_Generators_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.c nntool_output/networkModel.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler/LibTile.a  
In file included from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_BasicKernels_SQ8.h:21,
                 from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.h:21,
                 from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.c:21:
/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/../CNN_Libraries/CNN_Copy.h:23:10: fatal error: CNN_FloatType.h: No such file or directory
   23 | #include "CNN_FloatType.h"
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [common/model_rules.mk:56: BUILD_MODEL_SQ8BIT/GenTile] Error 1

I think this problem is caused by the large span of changing gap_sdkv3.9 to gap_sdkv4.22 (some file location may already changed). I wonder if there are any solutions? Or I can just modify the pulp-dronet's makefile by my own?

I try to solve this problem by my own and do make some progresses. If you have any suggestiones, pls tell me.