参照readme, 使用funasr推理，语音识别不准确

[lizhao-8202]

• OS (e.g., Linux): Windows10 专业版
• FunASR Version (e.g., 1.0.0):1.1.4
• ModelScope Version (e.g., 1.11.0):1.15.0
• PyTorch Version (e.g., 2.0.0):2.4.0
• How you installed funasr (pip, source): pip
• Python version:3.11.7
• GPU (e.g., V100M32):Graphics 630
• CUDA/cuDNN version (e.g., cuda11.7):cuda 12.5.40
• Docker version (e.g., funasr-runtime-sdk-cpu-0.4.1)：物理机发布
• Any other relevant information:

参照readme测试，代码如下

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model_dir = "D:\\study\\ai\\models\\SenseVoiceSmall"

model = AutoModel(model=model_dir, trust_remote_code=True, device="cuda:0")

res = model.generate(
    input=f"{model.model_path}/example/zh.mp3",
    cache={},
    #此处language用过auto,一样识别不对
    language="zh", # "zh", "en", "yue", "ja", "ko", "nospeech"
    use_itn=True,
    batch_size=64,
)

text = rich_transcription_postprocess(res[0]["text"])
print(text)

输出结果不对，具体日志如下 Notice: ffmpeg is not installed. torchaudio is used to load audio If you want to use ffmpeg backend to load audio, please install it by: sudo apt install ffmpeg # ubuntu

brew install ffmpeg # mac

If you want use mossformer, lease install rotary_embedding_torch by: pip install -U rotary_embedding_torch If you want use mossformer, lease install rotary_embedding_torch by: pip install -U rotary_embedding_torch If you want use mossformer, lease install rotary_embedding_torch by: pip install -U rotary_embedding_torch If you want use mossformer, lease install rotary_embedding_torch by: pip install -U rotary_embedding_torch Loading remote code failed: model, No module named 'model' D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\funasr\train_utils\load_pretrained_model.py:38: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. src_state = torch.load(path, map_location=map_location) rtf_avg: 0.279: 100%|██████████| 1/1 [00:01<00:00, 1.58s/it] mination缥1001.0000.

[LauraGPT]

Maybe you could test it by cpu: device="cpu" And show me the res.

[lizhao-8202]

Maybe you could test it by cpu: device="cpu" And show me the res.

装了rotary_embedding_torch 。然后添加device="cpu" 和不添加这一句，都试了下情况 1、加 device="cpu"

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model_dir = "D:\\study\\ai\\models\\SenseVoiceSmall"

model = AutoModel(model=model_dir, trust_remote_code=False, device="cuda:0")

res = model.generate(
    input=f"{model.model_path}/example/zh.mp3",
    cache={},
    language="auto", # "zh", "en", "yue", "ja", "ko", "nospeech"
    use_itn=True,
    batch_size=64,
    device="cpu",
)
print(f"##########res:{res}")
text = rich_transcription_postprocess(res[0]["text"])
print(text)

日志如下： Notice: ffmpeg is not installed. torchaudio is used to load audio If you want to use ffmpeg backend to load audio, please install it by: sudo apt install ffmpeg # ubuntu

brew install ffmpeg # mac

D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\rotary_embedding_torch\rotary_embedding_torch.py:35: FutureWarning: torch.cuda.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cuda', args...) instead. @autocast(enabled = False) D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\rotary_embedding_torch\rotary_embedding_torch.py:262: FutureWarning: torch.cuda.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cuda', args...) instead. @autocast(enabled = False) D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\funasr\train_utils\load_pretrained_model.py:38: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. src_state = torch.load(path, map_location=map_location) rtf_avg: 0.282: 100%|██████████| 1/1 [00:01<00:00, 1.59s/it] ##########res:[{'key': 'zh', 'text': '蒟1 작성된11.0.<|en|>.<|en|>致..'}] 蒟1 작성된11.0..致..

2、不加device="cpu"

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model_dir = "D:\\study\\ai\\models\\SenseVoiceSmall"

model = AutoModel(model=model_dir, trust_remote_code=False, device="cuda:0")

res = model.generate(
    input=f"{model.model_path}/example/zh.mp3",
    cache={},
    language="auto", # "zh", "en", "yue", "ja", "ko", "nospeech"
    use_itn=True,
    batch_size=64,
    # device="cpu",
)
print(f"##########res:{res}")
text = rich_transcription_postprocess(res[0]["text"])
print(text)

日志如下： Notice: ffmpeg is not installed. torchaudio is used to load audio If you want to use ffmpeg backend to load audio, please install it by: sudo apt install ffmpeg # ubuntu

brew install ffmpeg # mac

D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\rotary_embedding_torch\rotary_embedding_torch.py:35: FutureWarning: torch.cuda.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cuda', args...) instead. @autocast(enabled = False) D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\rotary_embedding_torch\rotary_embedding_torch.py:262: FutureWarning: torch.cuda.amp.autocast(args...) is deprecated. Please use torch.amp.autocast('cuda', args...) instead. @autocast(enabled = False) D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\funasr\train_utils\load_pretrained_model.py:38: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. src_state = torch.load(path, map_location=map_location) rtf_avg: 0.282: 100%|██████████| 1/1 [00:01<00:00, 1.59s/it] ##########res:[{'key': 'zh', 'text': '1>0<|en|>。<|en|>.<|en|>致<|en|>'}] 1>0。.致

[LauraGPT]

model = AutoModel(model=model_dir, trust_remote_code=False, device="cuda:0")

model = AutoModel(model=model_dir, trust_remote_code=False, device="cpu")

[lizhao-8202]

model = AutoModel(model=model_dir, trust_remote_code=False, device="cuda:0")

model = AutoModel(model=model_dir, trust_remote_code=False, device="cpu") 改了。

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model_dir = "D:\study\ai\models\SenseVoiceSmall"

model = AutoModel(model=model_dir, trust_remote_code=False, device="cuda:0")

model = AutoModel(model=model_dir, trust_remote_code=False, device="cpu")

res = model.generate( input=f"{model.model_path}/example/zh.mp3", cache={}, language="auto", # "zh", "en", "yue", "ja", "ko", "nospeech" use_itn=True, batch_size=64,

device="cpu",

) print(f"##########res:{res}") text = rich_transcription_postprocess(res[0]["text"]) print(text)

日志如下：
Notice: ffmpeg is not installed. torchaudio is used to load audio
If you want to use ffmpeg backend to load audio, please install it by:
    sudo apt install ffmpeg # ubuntu
    # brew install ffmpeg # mac
D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\rotary_embedding_torch\rotary_embedding_torch.py:35: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  @autocast(enabled = False)
D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\rotary_embedding_torch\rotary_embedding_torch.py:262: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  @autocast(enabled = False)
D:\AppData\Local\anaconda3\envs\local-sensevoice\Lib\site-packages\funasr\train_utils\load_pretrained_model.py:38: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  src_state = torch.load(path, map_location=map_location)
rtf_avg: 0.279: 100%|██████████| 1/1 [00:01<00:00,  1.58s/it]
##########res:[{'key': 'zh', 'text': '！<町俵 <|en|>.<|ja|>.。'}]
！<町俵..。

Process finished with exit code 0

[lizhao-8202]

貌似每执行一次，返回的结果还不一样。

[LauraGPT]

As the notice, maybe you could debug it by set weights_only=True in https://github.com/modelscope/FunASR/blob/main/funasr/train_utils/load_pretrained_model.py#L38

Make sure you install funasr by source code as the docs: https://github.com/modelscope/FunASR?tab=readme-ov-file#installation

[LauraGPT]

Please make sure torch<=2.3

[lizhao-8202]

Please make sure torch<=2.3

好的。感谢。我试下看

[lizhao-8202]

Please make sure torch<=2.3 换成了2.3.1的貌似正常了，换torch版本过程后会报错，需要重新安装torchaudio，funasr