search

GreenWaves-Technologies / gap_sdk

SDK for Greenwaves Technologies' GAP8 IoT Application Processor
https://greenwaves-technologies.com/en/gap8-the-internet-of-things-iot-application-processor/
Apache License 2.0
137 stars 75 forks source link

NNTOOL: unified_quantization_handler - indicating change of Gemm_31 output from c to chw order - rerun adjust command #371

Closed ChenMathilda closed 1 year ago

ChenMathilda commented 1 year ago

Hi, I used gap_sdk4.22.0 to quantify pulp-dronet-v2, but I met this error:

$ make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatMul_Conv_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8  -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
rm -f BUILD_MODEL_SQ8BIT/GenTile
rm -f -rf BUILD_MODEL_SQ8BIT
rm -f -rf nntool_output
mkdir BUILD_MODEL_SQ8BIT
cp nntool_input/models_onnx/model_original_ok.onnx BUILD_MODEL_SQ8BIT/network.onnx
echo "GENERATING NNTOOL STATE FILE"
GENERATING NNTOOL STATE FILE
echo BUILD_MODEL_SQ8BIT
BUILD_MODEL_SQ8BIT
nntool -s nntool_input/nntool_scripts/nntool_script_deployment BUILD_MODEL_SQ8BIT/network.onnx -q
settings - set log level to INFO
log_level - was: 'INFO'
now: 'INFO'
open - opening graph file BUILD_MODEL_SQ8BIT/network.onnx load_quantization = True
__init__ - unable to determine batch dimension. if the graph fails to import properly set it to 1 or a variable.
nngraph - update graph dimensions
nngraph - update graph dimensions
debug - was: False
now: True
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
nngraph - update graph dimensions
move_node_up - Node Sigmoid_36 cannot be moved
fuse_gap_convs - fusing nodes Conv_0,MaxPool_1 into Conv_0_fusion
fuse_gap_convs - fusing nodes Conv_2,Clip_4 into Conv_2_fusion
fuse_gap_convs - fusing nodes Conv_5,Clip_7 into Conv_5_fusion
fuse_gap_convs - fusing nodes Conv_8,Clip_9 into Conv_8_fusion
fuse_gap_convs - fusing nodes Conv_11,Clip_13 into Conv_11_fusion
fuse_gap_convs - fusing nodes Conv_14,Clip_16 into Conv_14_fusion
fuse_gap_convs - fusing nodes Conv_17,Clip_18 into Conv_17_fusion
fuse_gap_convs - fusing nodes Conv_20,Clip_22 into Conv_20_fusion
fuse_gap_convs - fusing nodes Conv_23,Clip_25 into Conv_23_fusion
fuse_gap_convs - fusing nodes Conv_26,Clip_27 into Conv_26_fusion
matcher - ++ fusion fuse_gap_convs modified graph
slice_to_split - replaced slice nodes Gather_33,Gather_35 with split node Gather_33_split
matcher - ++ fusion slice_to_split modified graph
insert_copies - inserting copy between Gather_33_reshape0:0 and output_1:0
matcher - ++ fusion insert_copies modified graph
remove_reshapes_before_linear - removing unnecessary reshape before linear Flatten_30
matcher - ++ fusion remove_reshapes_before_linear modified graph
fuse_op_activation - fusing nodes Add_28,Clip_29
matcher - ++ fusion fuse_op_activation_scale8 modified graph
move_node_up - Node Sigmoid_36 cannot be moved
matcher - ++ fusion scaled_match_group modified graph
move_node_up - Node Sigmoid_36 cannot be moved
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
input_norm_func - was: ''
now: 'x:x/255'
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
nngraph - update graph dimensions
/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/quantization/multiplicative/quantizers/filter_mult.py:241: RuntimeWarning: divide by zero encountered in log2
  biases_bits = np.ceil(np.log2(np.abs(biases_q.quantize(biases_node.dqvalue))))
unified_quantization_handler - indicating change of Gemm_31 input from c, out_cin_c, out_c to chw, out_cin_chw, out_c order - rerun adjust command
unified_quantization_handler - indicating change of Gemm_31 output from c to chw order - rerun adjust command
nngraph - update graph dimensions
nngraph - update graph dimensions
aquant - Quantization set. Use qshow command to see it.
nngraph - update graph dimensions
......

Error 1:

/gap_sdk/tools/nntool/nntool/quantization/multiplicative/quantizers/filter_mult.py:241: RuntimeWarning: divide by zero encountered in log2
  biases_bits = np.ceil(np.log2())

For the first question, I change np.abs(biases_q.quantize(biases_node.dqvalue)) to np.abs(biases_q.quantize(biases_node.dqvalue))+1-e5. It does work, but I'm not sure it's right.

Error 2:

unified_quantization_handler - indicating change of Gemm_31 input from c, out_cin_c, out_c to chw, out_cin_chw, out_c order - rerun adjust command
unified_quantization_handler - indicating change of Gemm_31 output from c to chw order - rerun adjust command

I guess that the input parameter name of the new version of Gemm has changed or the input dimension of Gemm changes , which cause this problem. But I don't know how and where to modify it.

sousoux commented 1 year ago

First correction looks sensible. Will take a look. Second should be handled automatically or rerun the adjust command.