CAM++ onnx infer与pytorch infer结果不一致

[njzheng]

运行模型

# Please install onnx in your python environment before
python speakerlab/bin/export_speaker_embedding_onnx.py \
    --experiment_path ./exp \
    --model_id iic/speech_campplus_sv_zh_en_16k-common_advanced \
    --target_onnx_file ./exp/campp.onnx

修改speakerlab/bin/export_speaker_embedding_onnx.py 同时令export_onnx_file返回dummy_input

def main():
    args = get_args()
    logger.info(f"{args}")

    model_id = args.model_id
    experiment_path = args.experiment_path
    target_onnx_file = args.target_onnx_file

    if model_id is not None:
        speaker_embedding_model = build_model_from_modelscope_id(
            model_id, experiment_path
        )
    else:
        speaker_embedding_model = build_model_from_custom_work_path(
            experiment_path
        )

    logger.info(f"Load speaker embedding finished, export to onnx")
    inputs = export_onnx_file(speaker_embedding_model, target_onnx_file)

    with torch.no_grad():
        res0 = speaker_embedding_model(inputs)

    import ipdb; ipdb.set_trace()
    ort_sess = ort.InferenceSession(target_onnx_file)
    res1 = ort_sess.run(None, {'feature': inputs.numpy()})[0]

    cos = torch.nn.CosineSimilarity(dim=1, eps=1e-6)
    print(cos(res0, res1))

结果cos 不是1,

ipdb> res0.sum()
tensor(15.4438)
ipdb> res1.sum()
tensor(16.6555)
ipdb> cos = nn.CosineSimilarity(dim=1, eps=1e-6)
*** NameError: name 'nn' is not defined
ipdb> cos = torch.nn.CosineSimilarity(dim=1, eps=1e-6)
ipdb> cos(res0, res1)
tensor([0.9864])

[GeekOrangeLuYao]

首先您提供的代码我这里直接运行会显示一个类型错误，所以我加了一句res1 = torch.from_numpy(res1)

def main():
    args = get_args()
    logger.info(f"{args}")

    model_id = args.model_id
    experiment_path = args.experiment_path
    target_onnx_file = args.target_onnx_file
    if model_id is not None:
        speaker_embedding_model = build_model_from_modelscope_id(
            model_id, experiment_path
        )
    else:
        speaker_embedding_model = build_model_from_custom_work_path(
            experiment_path
        )

    logger.info(f"Load speaker embedding finished, export to onnx")
    inputs = export_onnx_file(speaker_embedding_model, target_onnx_file)

    with torch.no_grad():
        res0 = speaker_embedding_model(inputs)
    ort_sess = ort.InferenceSession(target_onnx_file)
    res1 = ort_sess.run(None, {'feature': inputs.numpy()})[0]
    res1 = torch.from_numpy(res1) # Here, convert it to torch

    cos = torch.nn.CosineSimilarity(dim=1, eps=1e-6)
    print(cos(res0, res1))

我用上面这一段代码运行结果是1.0，我不确定您在pdb的时候是否对res0 / res1进行了修改？下面是我打印的信息：

$ python speakerlab/bin/export_speaker_embedding_onnx.py --experiment_path ./exp --model_id iic/speech_campplus_sv_zh_en_16k-common_advanced --target_onnx_file ./exp/campp.onnx
2024-09-30 11:10:36,272 - modelscope - INFO - PyTorch version 1.13.0 Found.
2024-09-30 11:10:36,274 - modelscope - INFO - Loading ast index from /Users/shuli/.cache/modelscope/ast_indexer
2024-09-30 11:10:36,435 - modelscope - INFO - Loading done! Current index file version is 1.9.3, with md5 a46a1afe0558f842902cf15497f2be3f and a total number of 943 components indexed
2024-09-30 11:10:37,950 - INFO: Namespace(experiment_path='./exp', model_id='iic/speech_campplus_sv_zh_en_16k-common_advanced', target_onnx_file='./exp/campp.onnx')
2024-09-30 11:10:37,950 - INFO: Build model from modelscope model_id: iic/speech_campplus_sv_zh_en_16k-common_advanced
2024-09-30 11:10:38,750 - modelscope - INFO - Use user-specified model revision: v1.0.0
2024-09-30 11:10:39,301 - INFO: Load speaker embedding finished, export to onnx
2024-09-30 11:10:44,655 - INFO: Export model onnx to ./exp/campp.onnx finished
tensor([1.])

考虑到inputs是一个随机的tensor，我这里运行了5次，但是结果是一致的都是1.0，或许您可以提供更多信息？我这里无法复现您的这个问题。

[njzheng]

可以提供下你funasr， pytorch， modelscope的版本吗？ 我这边是 torch.version '2.3.1+cu121' funasr.version ’1.1.3 modelscope.version '1.16.1'

结果如下，不是1

2024-09-30 11:16:16,474 - INFO: Namespace(experiment_path='./exp', model_id='iic/speech_campplus_sv_zh_en_16k-common_advanced', target_onnx_file='./exp/campp.onnx')
2024-09-30 11:16:16,474 - INFO: Build model from modelscope model_id: iic/speech_campplus_sv_zh_en_16k-common_advanced
2024-09-30 11:16:16,480 - modelscope - WARNING - Authentication has expired, please re-login with modelscope login --token "YOUR_SDK_TOKEN" if you need to access private models or datasets.
2024-09-30 11:16:19,791 - modelscope - INFO - Use user-specified model revision: v1.0.0
2024-09-30 11:16:20,964 - INFO: Load speaker embedding finished, export to onnx
2024-09-30 11:16:40,080 - INFO: Export model onnx to ./exp/campp.onnx finished
tensor(16.6841)
tensor(18.2456)
tensor([0.9881])

[njzheng]

应该pytorch版本问题，1.13可以

[GeekOrangeLuYao]

后续我们会check相关的版本带来的mismatch，感谢您的issue