Open polarispw opened 1 year ago
上述问题可以通过拉取支持fp16模型转换的最新版本解决,使用半精度的onnx模型直接转换为tnn模型。 但是转换出的tnn模型(Bert-base-uncased from huggingface)在TNNTest中无法运行:
/TNNTest -mp Bert_simp_half.tnnproto -dt=ARM -ic=5 -wc=1 -pr=NORMAL <
2023-03-27 14:59:42 773: E source/tnn/optimizer/graph_matcher/ir.cc:584 the graph is not connected.
E/tnn: virtual tnn::Status tnn::optimizer::NetOptimizerConvertMatMulToConv::Optimize(tnn::NetStructure *, tnn::NetResource *) [File source/tnn/optimizer/net_optimizer_convert_matmul_to_conv.cc][Line 77] code: 0x6000 msg: source/tnn/optimizer/graph_matcher/ir.cc:584 the graph is not connected.E/tnn: static tnn::DimsVector tnn::DimsFunctionUtils::StrideSlice(const tnn::DimsVector, tnn::DimsVector &, tnn::DimsVector &, const tnn::DimsVector, const tnn::DimsVector, tnn::Status *) [File source/tnn/utils/dims_function_utils.cc][Line 164] StrideSliceV2Layer param of axes, ends, strides size is invalid
E/tnn: virtual tnn::Status tnn::CpuSubLayerAcc::Calculate(const std::vector<Blob *> &, const std::vector<void *> &, const std::vector<DimsVector> &, tnn::Blob *) [File source/tnn/device/cpu/acc/cpu_sub_layer_acc.cc][Line 48] Error: CpuSubLayerAcc don't support data type: 1
E/tnn: virtual tnn::Status tnn::DefaultNetwork::Forward() [File source/tnn/core/default_network.cc][Line 603] Forward error code: 0x4000 msg: Error: CpuSubLayerAcc don't support data type, exit
E/tnn: bool tnn::test::CheckResult(std::string, tnn::Status) [File test/test.cc][Line 348] create instance failed: code: 0x4000 msg: Error: CpuSubLayerAcc don't support data type
@gttiankai
1. 使用环境(environment)
OS Version:
2. Github版本
commit(optional):
3. 详细描述bug 情况 (Describe the bug) A clear and concise description of what the bug is.
original model: ONNX 我在使用最新版镜像的docker对模型进行转换时遇到了fp16转换不成功的问题(debug参数开启后转换工具的输出是Succeed,但是用netron查看模型结果仍为fp32)
4. 日志(Log)
---------- convert model, please wait a moment ----------
Converter ONNX to TNN Model...
Converter ONNX to TNN check_onnx_dim...
Converter ONNX to TNN check_onnx_dim...
Model will be converted to FP16!!! // 这行是为了看half是否被启用自己加的 The onnx2tnn command:python3 onnx2tnn.py /workspace/matmul_linear1.onnx -version=202303210906 -optimize=0 -half=1 -o /workspace
E/tnn: onnx2tnn_convert [File /opt/TNN/tools/onnx2tnn/onnx-converter/onnx2tnn_convert.cc][Line 153] The input_info
0.----onnx version:1.6.0
algo_optimize 0
onnx_net_opt_path /workspace/matmul_linear1.onnx
2.----onnx2tnn: /workspace/matmul_linear1.onnx
3.----onnx2tnn status: 0
D/tnn: OnnxExtractBlobWeights [File /opt/TNN/tools/onnx2tnn/src/core/onnx2tnn.cc][Line 388] weight = linear.bias
D/tnn: OnnxExtractBlobWeights [File /opt/TNN/tools/onnx2tnn/src/core/onnx2tnn.cc][Line 388] weight = onnx::MatMul_6
D/tnn: TNNWriteProto [File /opt/TNN/tools/onnx2tnn/src/core/onnx2tnn.cc][Line 247] input_blob_shape dim_size: 3
D/tnn: TNNWriteProto [File /opt/TNN/tools/onnx2tnn/src/core/onnx2tnn.cc][Line 330] node:/linear/MatMul_output_0 onnx:MatMul -> tnn:MatMul
D/tnn: TNNWriteProto [File /opt/TNN/tools/onnx2tnn/src/core/onnx2tnn.cc][Line 330] node:output_1 onnx:Add -> tnn:Add
WriteTensorData [Line 160] tersor (onnx::MatMul_6) data type: 1 item_size: 1179648
WriteTensorData [Line 160] tersor (linear.bias) data type: 1 item_size: 1536
WriteTensorData [Line 160] tersor (linear.bias) data type: 1 item_size: 1536
D/tnn: Interpret [File /opt/TNN/source/tnn/interpreter/tnn/model_interpreter.cc][Line 103] model params md5: dc0cb66c1ba4de01cd48e0eea9107e00
D/tnn: Interpret [File /opt/TNN/source/tnn/interpreter/tnn/model_interpreter.cc][Line 103] model params md5: e0266ddf9b4f40ea35f25d0b443827e7
D/tnn: CpuSupportFp16 [File /opt/TNN/source/tnn/utils/cpu_utils.cc][Line 356] CpuUtils::CpuSupportFp16, TNN_ARM82 is off, fp16arith = 0.
D/tnn: Init [File /opt/TNN/source/tnn/core/default_network.cc][Line 86] support fp 16: 0
D/tnn: CpuSupportFp16 [File /opt/TNN/source/tnn/utils/cpu_utils.cc][Line 356] CpuUtils::CpuSupportFp16, TNN_ARM82 is off, fp16arith = 0.
D/tnn: ReshapeLayers [File /opt/TNN/source/tnn/core/default_network.cc][Line 711] ReshapeLayers Output Shape: [name: /linear/MatMul_output_0 data type: 0 shape: [ 16 128 1536 ]]
D/tnn: ReshapeLayers [File /opt/TNN/source/tnn/core/default_network.cc][Line 711] ReshapeLayers Output Shape: [name: output_1 data type: 0 shape: [ 16 128 1536 ]]
D/tnn: Forward [File /opt/TNN/source/tnn/core/default_network.cc][Line 600] layer name: /linear/MatMul, forward result: 0
D/tnn: Forward [File /opt/TNN/source/tnn/core/default_network.cc][Line 601] Output Shape: [name: /linear/MatMul_output_0 data type: 0 shape: [ 16 128 1536 ]]
D/tnn: Forward [File /opt/TNN/source/tnn/core/default_network.cc][Line 600] layer name: /linear/Add, forward result: 0
D/tnn: Forward [File /opt/TNN/source/tnn/core/default_network.cc][Line 601] Output Shape: [name: output_1 data type: 0 shape: [ 16 128 1536 ]]
Converter ONNX to TNN model succeed!