search

sophgo / tpu-mlir

Machine learning compiler based on MLIR for Sophgo TPU.
Other
611 stars 153 forks source link

BM1684x导出tinyClip的text_encoder的fp16模型在model_transform工具中结果符合预期，但model_deploy中结果错误 #190

Closed XiaotaoChen closed 1 month ago

XiaotaoChen commented 2 months ago

背景

  1. 我们正在基于bm1684x部署各类多模态模型，目前在尝试简单的tinyclip模型。正确导出了对应的vision encoder，text encoder对应的onnx模型；
  2. 使用tpu-mlir工具正确导出了vison encoder的bmodel，并验证结果正确；
  3. 目前在导出text encoder时，结果有误。详见下面错误；
  4. tpu-mlir 镜像环境为：sophgo/tpuc_dev:tpu-mlir-1.8

model_transform入参和输出

入参 其中text_inp.npz 和 text_feat.npz为对应onnx模型输入和输出结果，已验证结果正确。

model_transform.py \
    --model_name tinyclip-vit-40M_text-sim \
    --model_def resource/tinyclip/onnx/tinyclip-vit-40M_text-sim.onnx \
    --test_input /code/engine-efficient-deploy/text_inp.npz \
    --test_result /code/engine-efficient-deploy/text_feat.npz \
    --mlir tinyclip-vit-40M_text-sim.mlir

输出较长，见最后

model_deploy入参和输出信息

入参

model_deploy.py \
    --mlir tinyclip-vit-40M_text-sim.mlir \
    --quantize F16 \
    --chip bm1684x \
    --test_input /code/engine-efficient-deploy/text_inp.npz \
    --test_reference /code/engine-efficient-deploy/text_feat.npz \
    --model tinyclip-vit-40M_text-sim_bm1684x_F16.bmodel

输出

***************************************************
F16 quantization ...
2024/09/14 17:52:07 - INFO : TPU-MLIR v1.8-20240529
[Running]: tpuc-opt tinyclip-vit-40M_text-sim.mlir --processor-assign="chip=bm1684x num_device=1 num_core=1" --processor-top-optimize --convert-top-to-tpu="mode=F16  asymmetric=False doWinograd=False ignore_f16_overflow=False q_group_size=0" --canonicalize --weight-fold -o tinyclip-vit-40M_text-sim_bm1684x_f16_tpu.mlir
The dir path of compiler_profile is "./"
bmcpu init: skip cpu_user_defined
Cannot open libusercpu.so, disable user cpu layer.
[Success]: tpuc-opt tinyclip-vit-40M_text-sim.mlir --processor-assign="chip=bm1684x num_device=1 num_core=1" --processor-top-optimize --convert-top-to-tpu="mode=F16  asymmetric=False doWinograd=False ignore_f16_overflow=False q_group_size=0" --canonicalize --weight-fold -o tinyclip-vit-40M_text-sim_bm1684x_f16_tpu.mlir
[CMD]: model_runner.py --input tinyclip-vit-40M_text-sim_in_f32.npz --model tinyclip-vit-40M_text-sim_bm1684x_f16_tpu.mlir --output tinyclip-vit-40M_text-sim_bm1684x_f16_tpu_outputs.npz 
[##################################################] 100%
[Running]: npz_tool.py compare tinyclip-vit-40M_text-sim_bm1684x_f16_tpu_outputs.npz /code/engine-efficient-deploy/text_feat.npz --tolerance 0.8,0.5 --except - -vv
compare 1197_MatMul:   0%|                                                                  | 0/1 [00:00<?, ?it/s][1197_MatMul                     ]  NOT_SIMLIAR [FAILED]
    (1, 512) float32 
    cosine_similarity      = 0.601506
    euclidean_similarity   = -0.009980
    sqnr_similarity        = 0.121413
top-k:
 idx-t  target  idx-r  ref
  143 3.4882812 143 4.319598
  16 3.4863281 16 4.317416
  139 0.8105469 346 0.8060569
  291 0.7705078 90 0.6902993
  422 0.5546875 139 0.62082314
  133 0.54296875 181 0.61267024
  153 0.5239258 424 0.55200005
  120 0.4609375 86 0.5519037
  151 0.4597168 334 0.53900564
  31 0.45239258 265 0.51891255
1 compared
0 passed
  0 equal, 0 close, 0 similar
1 failed
  0 not equal, 1 not similar
min_similiarity = (0.6015061736106873, -0.009980305361532382, 0.12141303159296513)
Target    tinyclip-vit-40M_text-sim_bm1684x_f16_tpu_outputs.npz
Reference /code/engine-efficient-deploy/text_feat.npz
npz compare FAILED.
compare 1197_MatMul: 100%|██████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 56.46it/s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/tools/model_deploy.py", line 391, in <module>
    lowering_patterns = tool.lowering()
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/tools/model_deploy.py", line 148, in lowering
    tool.validate_tpu_mlir()
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/tools/model_deploy.py", line 252, in validate_tpu_mlir
    f32_blobs_compare(self.tpu_npz, self.ref_npz, self.tolerance, self.excepts)
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/utils/mlir_shell.py", line 363, in f32_blobs_compare
    _os_system(cmd)
  File "/usr/local/lib/python3.10/dist-packages/tpu_mlir/python/utils/mlir_shell.py", line 55, in _os_system
    raise RuntimeError("[!Error]: {}".format(cmd_str))
RuntimeError: [!Error]: npz_tool.py compare tinyclip-vit-40M_text-sim_bm1684x_f16_tpu_outputs.npz /code/engine-efficient-deploy/text_feat.npz --tolerance 0.8,0.5 --except - -vv

model_transform输出信息

***************************************************
2024/09/14 17:42:45 - INFO : TPU-MLIR v1.8-20240529
2024/09/14 17:42:45 - INFO : 
         _____________________________________________________ 
        | preprocess:                                           |
        |   (x - mean) * scale                                  |
        '-------------------------------------------------------'
  config Preprocess args : 
        resize_dims           : same to net input dims
        keep_aspect_ratio     : False
        keep_ratio_mode       : letterbox
        pad_value             : 0
        pad_type              : center
        --------------------------
        mean                  : [0.0, 0.0, 0.0]
        scale                 : [1.0, 1.0, 1.0]
        --------------------------
        pixel_format          : bgr
        channel_format        : nchw

2024/09/14 17:42:45 - INFO : Input_shape assigned
2024/09/14 17:42:46 - INFO : WARNING: ConstantFolding failed.
2024/09/14 17:42:46 - INFO : ConstantFolding finished
2024/09/14 17:42:46 - INFO : skip_fuse_bn:False
2024/09/14 17:42:47 - INFO : Onnxsim opt finished
2024/09/14 17:42:47 - INFO : WARNING: ConstantFolding failed.
2024/09/14 17:42:47 - INFO : ConstantFolding finished
2024/09/14 17:42:48 - INFO : Save mlir file: tinyclip-vit-40M_text-sim_origin.mlir
[Running]: tpuc-opt tinyclip-vit-40M_text-sim_origin.mlir --shape-infer --canonicalize --extra-optimize -o tinyclip-vit-40M_text-sim.mlir
[Success]: tpuc-opt tinyclip-vit-40M_text-sim_origin.mlir --shape-infer --canonicalize --extra-optimize -o tinyclip-vit-40M_text-sim.mlir
2024/09/14 17:42:48 - INFO : Mlir file generated:tinyclip-vit-40M_text-sim.mlir
[CMD]: model_runner.py --input tinyclip-vit-40M_text-sim_in_f32.npz --model /code/engine-efficient-deploy/resource/tinyclip/onnx/tinyclip-vit-40M_text-sim.onnx --output tinyclip-vit-40M_text-sim_ref_outputs.npz 
2024/09/14 17:42:48 - INFO : Saving tinyclip-vit-40M_text-sim_ref_outputs.npz
[CMD]: model_runner.py --input tinyclip-vit-40M_text-sim_in_f32.npz --model tinyclip-vit-40M_text-sim.mlir --output /code/engine-efficient-deploy/text_feat.npz 
[##################################################] 100%
2024/09/14 17:42:49 - INFO : Saving /code/engine-efficient-deploy/text_feat.npz
[Running]: npz_tool.py compare /code/engine-efficient-deploy/text_feat.npz tinyclip-vit-40M_text-sim_ref_outputs.npz --tolerance 0.99,0.99 --except - -vv
compare 1197_MatMul:  99%|████████████████████████████████████████████████████▋| 169/170 [00:00<00:00, 246.80it/s][/text_model/embeddings/token_embedding/Gather_output_0_Gather]        EQUAL [PASSED]
    (1, 7, 512) float32 
[/text_model/embeddings/Add_output_0_Add]        EQUAL [PASSED]
    (1, 7, 512) float32 
[/text_model/encoder/layers.0/layer_norm1/LayerNormalization_output_0_LayerNormalization]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/Mul_output_0_Mul]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/k_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.247305
[/text_model/encoder/layers.0/self_attn/Reshape_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.247305
[/text_model/encoder/layers.0/self_attn/Transpose_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.247305
[/text_model/encoder/layers.0/self_attn/v_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.302961
[/text_model/encoder/layers.0/self_attn/Reshape_1_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.302961
[/text_model/encoder/layers.0/self_attn/Transpose_1_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.302961
[/text_model/encoder/layers.0/self_attn/Reshape_2_output_0_Reshape]        CLOSE [PASSED]
    (1, 7, 8, 64) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/Transpose_2_output_0_Transpose]        CLOSE [PASSED]
    (1, 8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/Reshape_3_output_0_Reshape]        CLOSE [PASSED]
    (8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/Reshape_4_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.247305
[/text_model/encoder/layers.0/self_attn/Reshape_5_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.302961
[/text_model/encoder/layers.0/self_attn/Transpose_3_output_0_Transpose]      SIMILAR [PASSED]
    (8, 64, 7) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 132.247305
[/text_model/encoder/layers.0/self_attn/MatMul_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.0/self_attn/Add_output_0_Add]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.0/self_attn/Softmax_output_0_Softmax]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 5
[/text_model/encoder/layers.0/self_attn/MatMul_1_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/Reshape_8_output_0_Reshape]        CLOSE [PASSED]
    (1, 8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/Transpose_4_output_0_Transpose]        CLOSE [PASSED]
    (1, 7, 8, 64) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/Reshape_9_output_0_Reshape]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.0/self_attn/out_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.871292
[/text_model/encoder/layers.0/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 125.834484
[/text_model/encoder/layers.0/layer_norm2/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 128.617544
[/text_model/encoder/layers.0/mlp/fc1/Add_output_0_Add]        CLOSE [PASSED]
    (1, 7, 2048) float32 
    close order            = 3
[/text_model/encoder/layers.0/mlp/fc2/Add_output_0_Add]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.0/Add_1_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.305433
[/text_model/encoder/layers.1/layer_norm1/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.982935
[/text_model/encoder/layers.1/self_attn/Mul_output_0_Mul]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.803314
[/text_model/encoder/layers.1/self_attn/k_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.631281
[/text_model/encoder/layers.1/self_attn/Reshape_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.631281
[/text_model/encoder/layers.1/self_attn/Transpose_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.631281
[/text_model/encoder/layers.1/self_attn/v_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.890905
[/text_model/encoder/layers.1/self_attn/Reshape_1_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.890905
[/text_model/encoder/layers.1/self_attn/Transpose_1_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.890905
[/text_model/encoder/layers.1/self_attn/Reshape_2_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.803314
[/text_model/encoder/layers.1/self_attn/Transpose_2_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.803314
[/text_model/encoder/layers.1/self_attn/Reshape_3_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.803314
[/text_model/encoder/layers.1/self_attn/Reshape_4_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.631281
[/text_model/encoder/layers.1/self_attn/Reshape_5_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.890905
[/text_model/encoder/layers.1/self_attn/Transpose_3_output_0_Transpose]      SIMILAR [PASSED]
    (8, 64, 7) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.631281
[/text_model/encoder/layers.1/self_attn/MatMul_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.1/self_attn/Add_output_0_Add]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.1/self_attn/Softmax_output_0_Softmax]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 5
[/text_model/encoder/layers.1/self_attn/MatMul_1_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.1/self_attn/Reshape_8_output_0_Reshape]        CLOSE [PASSED]
    (1, 8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.1/self_attn/Transpose_4_output_0_Transpose]        CLOSE [PASSED]
    (1, 7, 8, 64) float32 
    close order            = 3
[/text_model/encoder/layers.1/self_attn/Reshape_9_output_0_Reshape]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.1/self_attn/out_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.076061
[/text_model/encoder/layers.1/Add_output_0_Add]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.1/layer_norm2/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.945684
[/text_model/encoder/layers.1/mlp/fc1/Add_output_0_Add]        CLOSE [PASSED]
    (1, 7, 2048) float32 
    close order            = 3
[/text_model/encoder/layers.1/mlp/fc2/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.392603
[/text_model/encoder/layers.1/Add_1_output_0_Add]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.2/layer_norm1/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 122.034874
[/text_model/encoder/layers.2/self_attn/Mul_output_0_Mul]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.2/self_attn/k_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.043652
[/text_model/encoder/layers.2/self_attn/Reshape_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.043652
[/text_model/encoder/layers.2/self_attn/Transpose_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.043652
[/text_model/encoder/layers.2/self_attn/v_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.531162
[/text_model/encoder/layers.2/self_attn/Reshape_1_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.531162
[/text_model/encoder/layers.2/self_attn/Transpose_1_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.531162
[/text_model/encoder/layers.2/self_attn/Reshape_2_output_0_Reshape]        CLOSE [PASSED]
    (1, 7, 8, 64) float32 
    close order            = 3
[/text_model/encoder/layers.2/self_attn/Transpose_2_output_0_Transpose]        CLOSE [PASSED]
    (1, 8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.2/self_attn/Reshape_3_output_0_Reshape]        CLOSE [PASSED]
    (8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.2/self_attn/Reshape_4_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.043652
[/text_model/encoder/layers.2/self_attn/Reshape_5_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.531162
[/text_model/encoder/layers.2/self_attn/Transpose_3_output_0_Transpose]      SIMILAR [PASSED]
    (8, 64, 7) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 123.043652
[/text_model/encoder/layers.2/self_attn/MatMul_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.2/self_attn/Add_output_0_Add]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 5
[/text_model/encoder/layers.2/self_attn/Softmax_output_0_Softmax]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 5
[/text_model/encoder/layers.2/self_attn/MatMul_1_output_0_MatMul]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.296341
[/text_model/encoder/layers.2/self_attn/Reshape_8_output_0_Reshape]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.296341
[/text_model/encoder/layers.2/self_attn/Transpose_4_output_0_Transpose]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.296341
[/text_model/encoder/layers.2/self_attn/Reshape_9_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.296341
[/text_model/encoder/layers.2/self_attn/out_proj/Add_output_0_Add]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.2/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.340805
[/text_model/encoder/layers.2/layer_norm2/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.745853
[/text_model/encoder/layers.2/mlp/fc1/Add_output_0_Add]        CLOSE [PASSED]
    (1, 7, 2048) float32 
    close order            = 3
[/text_model/encoder/layers.2/mlp/fc2/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.976542
[/text_model/encoder/layers.2/Add_1_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.311089
[/text_model/encoder/layers.3/layer_norm1/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.602461
[/text_model/encoder/layers.3/self_attn/Mul_output_0_Mul]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 122.215672
[/text_model/encoder/layers.3/self_attn/k_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.335926
[/text_model/encoder/layers.3/self_attn/Reshape_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.335926
[/text_model/encoder/layers.3/self_attn/Transpose_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.335926
[/text_model/encoder/layers.3/self_attn/v_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.158535
[/text_model/encoder/layers.3/self_attn/Reshape_1_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.158535
[/text_model/encoder/layers.3/self_attn/Transpose_1_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.158535
[/text_model/encoder/layers.3/self_attn/Reshape_2_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 122.215672
[/text_model/encoder/layers.3/self_attn/Transpose_2_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 122.215672
[/text_model/encoder/layers.3/self_attn/Reshape_3_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 122.215672
[/text_model/encoder/layers.3/self_attn/Reshape_4_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.335926
[/text_model/encoder/layers.3/self_attn/Reshape_5_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.158535
[/text_model/encoder/layers.3/self_attn/Transpose_3_output_0_Transpose]      SIMILAR [PASSED]
    (8, 64, 7) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.335926
[/text_model/encoder/layers.3/self_attn/MatMul_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.3/self_attn/Add_output_0_Add]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.3/self_attn/Softmax_output_0_Softmax]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 5
[/text_model/encoder/layers.3/self_attn/MatMul_1_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.3/self_attn/Reshape_8_output_0_Reshape]        CLOSE [PASSED]
    (1, 8, 7, 64) float32 
    close order            = 3
[/text_model/encoder/layers.3/self_attn/Transpose_4_output_0_Transpose]        CLOSE [PASSED]
    (1, 7, 8, 64) float32 
    close order            = 3
[/text_model/encoder/layers.3/self_attn/Reshape_9_output_0_Reshape]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.3/self_attn/out_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 124.409218
[/text_model/encoder/layers.3/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.318012
[/text_model/encoder/layers.3/layer_norm2/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.781590
[/text_model/encoder/layers.3/mlp/fc1/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 2048) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 120.757675
[/text_model/encoder/layers.3/mlp/fc2/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.326403
[/text_model/encoder/layers.3/Add_1_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.251751
[/text_model/encoder/layers.4/layer_norm1/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 118.705044
[/text_model/encoder/layers.4/self_attn/Mul_output_0_Mul]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.250296
[/text_model/encoder/layers.4/self_attn/k_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.047283
[/text_model/encoder/layers.4/self_attn/Reshape_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.047283
[/text_model/encoder/layers.4/self_attn/Transpose_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.047283
[/text_model/encoder/layers.4/self_attn/v_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 118.241129
[/text_model/encoder/layers.4/self_attn/Reshape_1_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 118.241129
[/text_model/encoder/layers.4/self_attn/Transpose_1_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 118.241129
[/text_model/encoder/layers.4/self_attn/Reshape_2_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.250296
[/text_model/encoder/layers.4/self_attn/Transpose_2_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.250296
[/text_model/encoder/layers.4/self_attn/Reshape_3_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.250296
[/text_model/encoder/layers.4/self_attn/Reshape_4_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.047283
[/text_model/encoder/layers.4/self_attn/Reshape_5_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 118.241129
[/text_model/encoder/layers.4/self_attn/Transpose_3_output_0_Transpose]      SIMILAR [PASSED]
    (8, 64, 7) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.047283
[/text_model/encoder/layers.4/self_attn/MatMul_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.4/self_attn/Add_output_0_Add]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.4/self_attn/Softmax_output_0_Softmax]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 5
[/text_model/encoder/layers.4/self_attn/MatMul_1_output_0_MatMul]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.040154
[/text_model/encoder/layers.4/self_attn/Reshape_8_output_0_Reshape]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.040154
[/text_model/encoder/layers.4/self_attn/Transpose_4_output_0_Transpose]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.040154
[/text_model/encoder/layers.4/self_attn/Reshape_9_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 121.040154
[/text_model/encoder/layers.4/self_attn/out_proj/Add_output_0_Add]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.4/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.273962
[/text_model/encoder/layers.4/layer_norm2/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 118.994026
[/text_model/encoder/layers.4/mlp/fc1/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 2048) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.015507
[/text_model/encoder/layers.4/mlp/fc2/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.687159
[/text_model/encoder/layers.4/Add_1_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.128393
[/text_model/encoder/layers.5/layer_norm1/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.588110
[/text_model/encoder/layers.5/self_attn/Mul_output_0_Mul]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.337324
[/text_model/encoder/layers.5/self_attn/k_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.261370
[/text_model/encoder/layers.5/self_attn/Reshape_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.261370
[/text_model/encoder/layers.5/self_attn/Transpose_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.261370
[/text_model/encoder/layers.5/self_attn/v_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.074347
[/text_model/encoder/layers.5/self_attn/Reshape_1_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.074347
[/text_model/encoder/layers.5/self_attn/Transpose_1_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.074347
[/text_model/encoder/layers.5/self_attn/Reshape_2_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.337324
[/text_model/encoder/layers.5/self_attn/Transpose_2_output_0_Transpose]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.337324
[/text_model/encoder/layers.5/self_attn/Reshape_3_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.337324
[/text_model/encoder/layers.5/self_attn/Reshape_4_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.261370
[/text_model/encoder/layers.5/self_attn/Reshape_5_output_0_Reshape]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.074347
[/text_model/encoder/layers.5/self_attn/Transpose_3_output_0_Transpose]      SIMILAR [PASSED]
    (8, 64, 7) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.261370
[/text_model/encoder/layers.5/self_attn/MatMul_output_0_MatMul]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 3
[/text_model/encoder/layers.5/self_attn/Add_output_0_Add]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.5/self_attn/Softmax_output_0_Softmax]        CLOSE [PASSED]
    (8, 7, 7) float32 
    close order            = 4
[/text_model/encoder/layers.5/self_attn/MatMul_1_output_0_MatMul]      SIMILAR [PASSED]
    (8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.236366
[/text_model/encoder/layers.5/self_attn/Reshape_8_output_0_Reshape]      SIMILAR [PASSED]
    (1, 8, 7, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.236366
[/text_model/encoder/layers.5/self_attn/Transpose_4_output_0_Transpose]      SIMILAR [PASSED]
    (1, 7, 8, 64) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.236366
[/text_model/encoder/layers.5/self_attn/Reshape_9_output_0_Reshape]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 117.236366
[/text_model/encoder/layers.5/self_attn/out_proj/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 120.930843
[/text_model/encoder/layers.5/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.127554
[/text_model/encoder/layers.5/layer_norm2/LayerNormalization_output_0_LayerNormalization]        CLOSE [PASSED]
    (1, 7, 512) float32 
    close order            = 3
[/text_model/encoder/layers.5/mlp/fc1/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 2048) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.856300
[/text_model/encoder/layers.5/mlp/fc2/Add_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 122.211542
[/text_model/encoder/layers.5/Add_1_output_0_Add]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 1.000000
    sqnr_similarity        = 135.012808
[/text_model/final_layer_norm/LayerNormalization_output_0_LayerNormalization]      SIMILAR [PASSED]
    (1, 7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.141827
[/text_model/Equal_1_output_0_Equal]        EQUAL [PASSED]
    (1, 7) float32 
[/text_model/ArgMax_output_0_ArgMax]        EQUAL [PASSED]
    (1,) float32 
[/text_model/Flatten_output_0_Flatten]      SIMILAR [PASSED]
    (7, 512) float32 
    cosine_similarity      = 1.000000
    euclidean_similarity   = 0.999999
    sqnr_similarity        = 119.141827
[/text_model/Gather_2_output_0_Gather]        CLOSE [PASSED]
    (1, 512) float32 
    close order            = 3
[1197_MatMul                     ]        CLOSE [PASSED]
    (1, 512) float32 
    close order            = 4
170 compared
170 passed
  4 equal, 50 close, 116 similar
0 failed
  0 not equal, 0 not similar
min_similiarity = (0.9999998211860657, 0.9999985987953269, 117.0743465423584)
Target    /code/engine-efficient-deploy/text_feat.npz
Reference tinyclip-vit-40M_text-sim_ref_outputs.npz
npz compare PASSED.
compare 1197_MatMul: 100%|█████████████████████████████████████████████████████| 170/170 [00:00<00:00, 249.00it/s]
[Success]: npz_tool.py compare /code/engine-efficient-deploy/text_feat.npz tinyclip-vit-40M_text-sim_ref_outputs.npz --tolerance 0.99,0.99 --except - -vv
charlesxzb commented 2 months ago

model_deploy阶段可以打开--compare_all获取所有层的对比信息，看看出错是在哪个节点上

XiaotaoChen commented 2 months ago

感谢，我们这边试了下，发现是因为在model_transform阶段也使用text_inp.npz和 text_feat.npz进行验证，这一步貌似会修改text_feat.npz的内容，保存每一层的输出信息；然后我们再拿这个text_feat.npz用在model_deploy中对齐，就报错了。使用最原始的仅包含输出tensor的text_feat.npz就能验证通过。

XiaotaoChen commented 1 month ago

model_deploy阶段可以打开--compare_all获取所有层的对比信息，看看出错是在哪个节点上

@charlesxzb 你好，两个新问题。

  1. 我们这边发现，虽然使用原始的text_feat.npz进行验证程序通过了。但看输出目录下的子目录tinyclip-vit-40M_text-sim_bm1684x_F16下有input_ref_data.dat, output_ref_data.dat数据，查看这里的数据，input_ref_data.dat结果和text_inp.npz一致，但output_ref_data.dat结果和text_feat.npz明显不一致，不知道为啥会验证通过；
  2. 我们在真实部署环境中使用text_inp.npz内容测试text_encoder模型发现输出结果不符合预期。输出结果和output_ref_data.dat一致；
  3. 综上所述，在我们部署环境中text_encoder结果没有对齐，问题应该可以通过output_ref_data.dat为啥和text_feat.npz不一致定位。应该是使用方式哪里有问题。麻烦解答一下。
XiaotaoChen commented 1 month ago

根据官方解答，是因为f16模式equal算子对比数值超出表示范围导致的。转成fp32模型或f16模型对于具体层使用fp32数据格式即可避免该问题。