rockchip-linux / rknn-toolkit2

BSD 3-Clause "New" or "Revised" License
874 stars 154 forks source link

模型转换中存在许多fuse_ops ,是否会影响模型推理? #274

Closed Eaglewzw closed 8 months ago

Eaglewzw commented 8 months ago
   在我进行模型推理过程中,我将可以正常进行推理ONNX模型转换成RKNN,发现无法成功推理,在打印的日志文件中,存在以下替换过程,我想询问是否存在类似以下的替换会导致模型推理结果不正确,如果是,是否存在一些解决办法?

I Save log info to: log.txt W init: rknn-toolkit2 version: 1.6.0+81f21f4d --> Config model done --> Loading model W load_onnx: It is recommended onnx opset 19, but your onnx model opset is 12! W load_onnx: Model converted from pytorch, 'opset_version' should be set 19 in torch.onnx.export for successful convert! Loading : 100%|████████████████████████████████████████████████| 191/191 [00:00<00:00, 10933.85it/s] W build: The dataset='./dataset.txt' is ignored because do_quantization = False! done --> Building model I base_optimize ... I base_optimize done. I I fold_constant ... I fold_constant done. I fold_constant remove nodes = ['Identity_0'] I I correct_ops ... I correct_ops done. I I fuse_ops ... I fuse_ops results: I replace_exswish: remove node = ['Sigmoid_2', 'Mul_3'], add node = ['Sigmoid_2_2swish'] I replace_exswish: remove node = ['Sigmoid_5', 'Mul_6'], add node = ['Sigmoid_5_2swish'] I replace_exswish: remove node = ['Sigmoid_8', 'Mul_9'], add node = ['Sigmoid_8_2swish'] I replace_exswish: remove node = ['Sigmoid_11', 'Mul_12'], add node = ['Sigmoid_11_2swish'] I replace_exswish: remove node = ['Sigmoid_14', 'Mul_15'], add node = ['Sigmoid_14_2swish'] I replace_exswish: remove node = ['Sigmoid_18', 'Mul_19'], add node = ['Sigmoid_18_2swish'] I replace_exswish: remove node = ['Sigmoid_21', 'Mul_22'], add node = ['Sigmoid_21_2swish'] I replace_exswish: remove node = ['Sigmoid_25', 'Mul_26'], add node = ['Sigmoid_25_2swish'] I replace_exswish: remove node = ['Sigmoid_29', 'Mul_30'], add node = ['Sigmoid_29_2swish'] I replace_exswish: remove node = ['Sigmoid_32', 'Mul_33'], add node = ['Sigmoid_32_2swish'] I replace_exswish: remove node = ['Sigmoid_35', 'Mul_36'], add node = ['Sigmoid_35_2swish'] I replace_exswish: remove node = ['Sigmoid_38', 'Mul_39'], add node = ['Sigmoid_38_2swish'] I replace_exswish: remove node = ['Sigmoid_41', 'Mul_42'], add node = ['Sigmoid_41_2swish'] I replace_exswish: remove node = ['Sigmoid_45', 'Mul_46'], add node = ['Sigmoid_45_2swish'] I replace_exswish: remove node = ['Sigmoid_48', 'Mul_49'], add node = ['Sigmoid_48_2swish'] I replace_exswish: remove node = ['Sigmoid_52', 'Mul_53'], add node = ['Sigmoid_52_2swish'] I replace_exswish: remove node = ['Sigmoid_55', 'Mul_56'], add node = ['Sigmoid_55_2swish'] I replace_exswish: remove node = ['Sigmoid_59', 'Mul_60'], add node = ['Sigmoid_59_2swish'] I replace_exswish: remove node = ['Sigmoid_62', 'Mul_63'], add node = ['Sigmoid_62_2swish'] I replace_exswish: remove node = ['Sigmoid_66', 'Mul_67'], add node = ['Sigmoid_66_2swish'] I replace_exswish: remove node = ['Sigmoid_70', 'Mul_71'], add node = ['Sigmoid_70_2swish'] I replace_exswish: remove node = ['Sigmoid_73', 'Mul_74'], add node = ['Sigmoid_73_2swish'] I replace_exswish: remove node = ['Sigmoid_76', 'Mul_77'], add node = ['Sigmoid_76_2swish'] I replace_exswish: remove node = ['Sigmoid_79', 'Mul_80'], add node = ['Sigmoid_79_2swish'] I replace_exswish: remove node = ['Sigmoid_82', 'Mul_83'], add node = ['Sigmoid_82_2swish'] I replace_exswish: remove node = ['Sigmoid_86', 'Mul_87'], add node = ['Sigmoid_86_2swish'] I replace_exswish: remove node = ['Sigmoid_89', 'Mul_90'], add node = ['Sigmoid_89_2swish'] I replace_exswish: remove node = ['Sigmoid_93', 'Mul_94'], add node = ['Sigmoid_93_2swish'] I replace_exswish: remove node = ['Sigmoid_96', 'Mul_97'], add node = ['Sigmoid_96_2swish'] I replace_exswish: remove node = ['Sigmoid_100', 'Mul_101'], add node = ['Sigmoid_100_2swish'] I replace_exswish: remove node = ['Sigmoid_103', 'Mul_104'], add node = ['Sigmoid_103_2swish'] I replace_exswish: remove node = ['Sigmoid_107', 'Mul_108'], add node = ['Sigmoid_107_2swish'] I replace_exswish: remove node = ['Sigmoid_110', 'Mul_111'], add node = ['Sigmoid_110_2swish'] I replace_exswish: remove node = ['Sigmoid_114', 'Mul_115'], add node = ['Sigmoid_114_2swish'] I replace_exswish: remove node = ['Sigmoid_117', 'Mul_118'], add node = ['Sigmoid_117_2swish'] I replace_exswish: remove node = ['Sigmoid_121', 'Mul_122'], add node = ['Sigmoid_121_2swish'] I replace_exswish: remove node = ['Sigmoid_125', 'Mul_126'], add node = ['Sigmoid_125_2swish'] I replace_exswish: remove node = ['Sigmoid_128', 'Mul_129'], add node = ['Sigmoid_128_2swish'] I replace_exswish: remove node = ['Sigmoid_131', 'Mul_132'], add node = ['Sigmoid_131_2swish'] I replace_exswish: remove node = ['Sigmoid_134', 'Mul_135'], add node = ['Sigmoid_134_2swish'] I replace_exswish: remove node = ['Sigmoid_137', 'Mul_138'], add node = ['Sigmoid_137_2swish'] I replace_exswish: remove node = ['Sigmoid_141', 'Mul_142'], add node = ['Sigmoid_141_2swish'] I replace_exswish: remove node = ['Sigmoid_144', 'Mul_145'], add node = ['Sigmoid_144_2swish'] I replace_exswish: remove node = ['Sigmoid_148', 'Mul_149'], add node = ['Sigmoid_148_2swish'] I replace_exswish: remove node = ['Sigmoid_152', 'Mul_153'], add node = ['Sigmoid_152_2swish'] I replace_exswish: remove node = ['Sigmoid_155', 'Mul_156'], add node = ['Sigmoid_155_2swish'] I replace_exswish: remove node = ['Sigmoid_162', 'Mul_163'], add node = ['Sigmoid_162_2swish'] I replace_exswish: remove node = ['Sigmoid_165', 'Mul_166'], add node = ['Sigmoid_165_2swish'] I replace_exswish: remove node = ['Sigmoid_171', 'Mul_172'], add node = ['Sigmoid_171_2swish'] I replace_exswish: remove node = ['Sigmoid_174', 'Mul_175'], add node = ['Sigmoid_174_2swish'] I replace_exswish: remove node = ['Sigmoid_177', 'Mul_178'], add node = ['Sigmoid_177_2swish'] I replace_exswish: remove node = ['Sigmoid_180', 'Mul_181'], add node = ['Sigmoid_180_2swish'] I replace_exswish: remove node = ['Sigmoid_183', 'Mul_184'], add node = ['Sigmoid_183_2swish'] I replace_exswish: remove node = ['Sigmoid_186', 'Mul_187'], add node = ['Sigmoid_186_2swish'] I replace_exswish: remove node = ['Sigmoid_190', 'Mul_191'], add node = ['Sigmoid_190_2swish'] I replace_exswish: remove node = ['Sigmoid_193', 'Mul_194'], add node = ['Sigmoid_193_2swish'] I replace_exswish: remove node = ['Sigmoid_199', 'Mul_200'], add node = ['Sigmoid_199_2swish'] I replace_exswish: remove node = ['Sigmoid_202', 'Mul_203'], add node = ['Sigmoid_202_2swish'] I replace_exswish: remove node = ['Sigmoid_205', 'Mul_206'], add node = ['Sigmoid_205_2swish'] I replace_exswish: remove node = ['Sigmoid_208', 'Mul_209'], add node = ['Sigmoid_208_2swish'] I replace_exswish: remove node = ['Sigmoid_211', 'Mul_212'], add node = ['Sigmoid_211_2swish'] I replace_exswish: remove node = ['Sigmoid_214', 'Mul_215'], add node = ['Sigmoid_214_2swish'] I replace_exswish: remove node = ['Sigmoid_218', 'Mul_219'], add node = ['Sigmoid_218_2swish'] I replace_exswish: remove node = ['Sigmoid_221', 'Mul_222'], add node = ['Sigmoid_221_2swish'] I replace_exswish: remove node = ['Sigmoid_225', 'Mul_226'], add node = ['Sigmoid_225_2swish'] I replace_exswish: remove node = ['Sigmoid_228', 'Mul_229'], add node = ['Sigmoid_228_2swish'] I replace_exswish: remove node = ['Sigmoid_231', 'Mul_232'], add node = ['Sigmoid_231_2swish'] I replace_exswish: remove node = ['Sigmoid_234', 'Mul_235'], add node = ['Sigmoid_234_2swish'] I replace_exswish: remove node = ['Sigmoid_237', 'Mul_238'], add node = ['Sigmoid_237_2swish'] I replace_exswish: remove node = ['Sigmoid_240', 'Mul_241'], add node = ['Sigmoid_240_2swish'] I replace_exswish: remove node = ['Sigmoid_244', 'Mul_245'], add node = ['Sigmoid_244_2swish'] I replace_exswish: remove node = ['Sigmoid_247', 'Mul_248'], add node = ['Sigmoid_247_2swish'] I replace_exswish: remove node = ['Sigmoid_251', 'Mul_252'], add node = ['Sigmoid_251_2swish'] I replace_exswish: remove node = ['Sigmoid_254', 'Mul_255'], add node = ['Sigmoid_254_2swish'] I replace_exswish: remove node = ['Sigmoid_257', 'Mul_258'], add node = ['Sigmoid_257_2swish'] I replace_exswish: remove node = ['Sigmoid_260', 'Mul_261'], add node = ['Sigmoid_260_2swish'] I replace_exswish: remove node = ['Sigmoid_263', 'Mul_264'], add node = ['Sigmoid_263_2swish'] I replace_exswish: remove node = ['Sigmoid_266', 'Mul_267'], add node = ['Sigmoid_266_2swish'] I replace_exswish: remove node = ['Sigmoid_270', 'Mul_271'], add node = ['Sigmoid_270_2swish'] I squeeze_to_4d_transpose: remove node = [], add node = ['Transpose_312_squeeze0', 'Transpose_312_squeeze1'] I squeeze_to_4d_sigmoid: remove node = [], add node = ['Sigmoid_313_0_squeeze0', 'Sigmoid_313_0_squeeze1'] I squeeze_to_4d_split: remove node = [], add node = ['Split_314_0_squeeze0', 'Split_314_0_squeeze1', 'Split_314_1_squeeze1', 'Split_314_2_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_316_0_squeeze0', 'Mul_316_0_squeeze1'] I squeeze_to_4d_add: remove node = [], add node = ['Add_318_0_squeeze0', 'Add_318_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_320_0_squeeze0', 'Mul_320_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_322_0_squeeze0', 'Mul_322_0_squeeze1'] I squeeze_to_4d_pow: remove node = [], add node = ['Pow_324_0_squeeze0', 'Pow_324_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_326_0_squeeze0', 'Mul_326_0_squeeze1'] I squeeze_to_4d_concat: remove node = [], add node = ['Concat_327_0_squeeze0', 'Concat_327_1_squeeze0', 'Concat_327_2_squeeze0', 'Concat_327_0_squeeze1'] I squeeze_to_4d_transpose: remove node = [], add node = ['Transpose_293_squeeze0', 'Transpose_293_squeeze1'] I squeeze_to_4d_sigmoid: remove node = [], add node = ['Sigmoid_294_0_squeeze0', 'Sigmoid_294_0_squeeze1'] I squeeze_to_4d_split: remove node = [], add node = ['Split_295_0_squeeze0', 'Split_295_0_squeeze1', 'Split_295_1_squeeze1', 'Split_295_2_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_297_0_squeeze0', 'Mul_297_0_squeeze1'] I squeeze_to_4d_add: remove node = [], add node = ['Add_299_0_squeeze0', 'Add_299_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_301_0_squeeze0', 'Mul_301_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_303_0_squeeze0', 'Mul_303_0_squeeze1'] I squeeze_to_4d_pow: remove node = [], add node = ['Pow_305_0_squeeze0', 'Pow_305_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_307_0_squeeze0', 'Mul_307_0_squeeze1'] I squeeze_to_4d_concat: remove node = [], add node = ['Concat_308_0_squeeze0', 'Concat_308_1_squeeze0', 'Concat_308_2_squeeze0', 'Concat_308_0_squeeze1'] I squeeze_to_4d_transpose: remove node = [], add node = ['Transpose_274_squeeze0', 'Transpose_274_squeeze1'] I squeeze_to_4d_sigmoid: remove node = [], add node = ['Sigmoid_275_0_squeeze0', 'Sigmoid_275_0_squeeze1'] I squeeze_to_4d_split: remove node = [], add node = ['Split_276_0_squeeze0', 'Split_276_0_squeeze1', 'Split_276_1_squeeze1', 'Split_276_2_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_278_0_squeeze0', 'Mul_278_0_squeeze1'] I squeeze_to_4d_add: remove node = [], add node = ['Add_280_0_squeeze0', 'Add_280_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_282_0_squeeze0', 'Mul_282_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_284_0_squeeze0', 'Mul_284_0_squeeze1'] I squeeze_to_4d_pow: remove node = [], add node = ['Pow_286_0_squeeze0', 'Pow_286_0_squeeze1'] I squeeze_to_4d_mul: remove node = [], add node = ['Mul_288_0_squeeze0', 'Mul_288_0_squeeze1'] I squeeze_to_4d_concat: remove node = [], add node = ['Concat_289_0_squeeze0', 'Concat_289_1_squeeze0', 'Concat_289_2_squeeze0', 'Concat_289_0_squeeze1'] I unsqueeze_to_4d_concat: remove node = [], add node = ['Concat_329_0_unsqueeze0', 'Concat_329_1_unsqueeze0', 'Concat_329_2_unsqueeze0', 'Concat_329_0_unsqueeze1'] I fuse_two_reshape: remove node = ['Reshape_311', 'Transpose_312_squeeze1', 'Sigmoid_313_0_squeeze1', 'Split_314_0_squeeze1', 'Mul_316_0_squeeze1', 'Add_318_0_squeeze1', 'Mul_320_0_squeeze1', 'Split_314_1_squeeze1', 'Mul_322_0_squeeze1', 'Pow_324_0_squeeze1', 'Mul_326_0_squeeze1', 'Split_314_2_squeeze1', 'Concat_327_0_squeeze1', 'Reshape_292', 'Transpose_293_squeeze1', 'Sigmoid_294_0_squeeze1', 'Split_295_0_squeeze1', 'Mul_297_0_squeeze1', 'Add_299_0_squeeze1', 'Mul_301_0_squeeze1', 'Split_295_1_squeeze1', 'Mul_303_0_squeeze1', 'Pow_305_0_squeeze1', 'Mul_307_0_squeeze1', 'Split_295_2_squeeze1', 'Concat_308_0_squeeze1', 'Reshape_273', 'Transpose_274_squeeze1', 'Sigmoid_275_0_squeeze1', 'Split_276_0_squeeze1', 'Mul_278_0_squeeze1', 'Add_280_0_squeeze1', 'Mul_282_0_squeeze1', 'Split_276_1_squeeze1', 'Mul_284_0_squeeze1', 'Pow_286_0_squeeze1', 'Mul_288_0_squeeze1', 'Split_276_2_squeeze1', 'Concat_289_0_squeeze1'] I swap_concat_axis_avoid_channel_concat: remove node = [], add node = ['Concat_329_swap_concat_reshape_i0_out', 'Concat_329_swap_concat_reshape_i1_out', 'Concat_329_swap_concat_reshape_i2_out', 'Concat_329_swap_concat_reshape_o0_out'] I remove_invalid_reshape: remove node = ['Sigmoid_313_0_squeeze0', 'Split_314_0_squeeze0', 'Mul_316_0_squeeze0', 'Add_318_0_squeeze0', 'Mul_320_0_squeeze0', 'Concat_327_0_squeeze0', 'Mul_322_0_squeeze0', 'Pow_324_0_squeeze0', 'Mul_326_0_squeeze0', 'Concat_327_1_squeeze0', 'Concat_327_2_squeeze0'] I fuse_two_reshape: remove node = ['Reshape_328'] I remove_invalid_reshape: remove node = ['Sigmoid_294_0_squeeze0', 'Split_295_0_squeeze0', 'Mul_297_0_squeeze0', 'Add_299_0_squeeze0', 'Mul_301_0_squeeze0', 'Concat_308_0_squeeze0', 'Mul_303_0_squeeze0', 'Pow_305_0_squeeze0', 'Mul_307_0_squeeze0', 'Concat_308_1_squeeze0', 'Concat_308_2_squeeze0'] I fuse_two_reshape: remove node = ['Reshape_309'] I remove_invalid_reshape: remove node = ['Sigmoid_275_0_squeeze0', 'Split_276_0_squeeze0', 'Mul_278_0_squeeze0', 'Add_280_0_squeeze0', 'Mul_282_0_squeeze0', 'Concat_289_0_squeeze0', 'Mul_284_0_squeeze0', 'Pow_286_0_squeeze0', 'Mul_288_0_squeeze0', 'Concat_289_1_squeeze0', 'Concat_289_2_squeeze0'] I fuse_two_reshape: remove node = ['Reshape_290', 'Concat_329_swap_concat_reshape_o0_out'] I fuse_two_reshape: remove node = ['Concat_329_2_unsqueeze0', 'Concat_329_1_unsqueeze0', 'Concat_329_0_unsqueeze0'] I fold_constant ... I fold_constant done. I fuse_ops done. I I sparse_weight ... I sparse_weight done. I I rknn building ... QQ20240114-150733