modelscope / FunASR

A Fundamental End-to-End Speech Recognition Toolkit and Open Source SOTA Pretrained Models, Supporting Speech Recognition, Voice Activity Detection, Text Post-processing etc.
https://www.funasr.com
Other
6.49k stars 688 forks source link

连续识别后内存不会释放 #921

Closed juvu closed 8 months ago

juvu commented 1 year ago

Ubuntu 环境,docker 24.0.5,最新2.0.1镜像,识别大型音频(300m以上)之后,内存不会释放,导致多次识别之后内存溢出

Dalaoyel commented 1 year ago

我也遇到了相同的问题,是在cpu识别的时候出现的。 调整max_single_segment_time 的最大值 可以减缓内存泄露速度

lyblsgo commented 1 year ago

我也遇到了相同的问题,是在cpu识别的时候出现的。 调整max_single_segment_time 的最大值 可以减缓内存泄露速度

Could you please provide more details about your problem, including the environment of the server and client, the audio used for testing, and the corresponding logs? Without being able to reproduce the issue, it would be difficult for us to troubleshoot and provide a solution.

Dalaoyel commented 1 year ago

我也遇到了相同的问题,是在cpu识别的时候出现的。 调整max_single_segment_time 的最大值 可以减缓内存泄露速度

Could you please provide more details about your problem, including the environment of the server and client, the audio used for testing, and the corresponding logs? Without being able to reproduce the issue, it would be difficult for us to troubleshoot and provide a solution.

OS: linux aarch64 Kylin Linux Advanced Server release V10 (Sword)

Python/C++ Version:3.7.5 Package Version: pytorch==1.13.1、 torchaudio==0.13.1、 modelscope==1.6.1、 funasr==0.7.5、

Model:speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch

command: from modelscope.pipelines import pipeline from modelscope.utils.constant import Tasks inference_pipeline = pipeline( task=Tasks.auto_speech_recognition, model='damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch', ) rec_result = inference_pipeline(audio_in=save_path,batch_size_token=1000, max_single_segment_time = 500 )

1.When I do not set max_single_segment_time, the memory grows fast and is not freed. 2.The average video is 20 minutes and is downloaded from Station bilibili 3.the top stat is Down here image The size of the initial loaded complete model is 2.3g

dd-guo commented 10 months ago

同样碰到类似问题, 初始内存2.3G, 识别一段时间后会涨到6G, 而且内存不断上涨

chenyangMl commented 8 months ago

同样碰到类似问题, 初始内存2.3G, 识别一段时间后会涨到6G, 而且内存不断上涨

你好,请问您这个问题解决了?

dd-guo commented 8 months ago

同样碰到类似问题, 初始内存2.3G, 识别一段时间后会涨到6G, 而且内存不断上涨

你好,请问您这个问题解决了?

是modelscope的bug导致的, funasr返回结果ok的.

lyblsgo commented 8 months ago

Try the newest version. If the issue persists, please reopen the issue and provide detailed steps to reproduce, as well as server and client logs.

zhushuaiCoding commented 4 months ago

请问解决了吗,我的用法如下,但是运行结束之后内存还是没有完全回收,导致下次再运行就内存溢出。有没有大佬解决一下



vad_model="fsmn-vad", vad_model_revision="v2.0.4",
vad_kwargs={"max_single_segment_time": 30000},
punc_model="ct-punc-c", punc_model_revision="v2.0.4",
spk_model="cam++", spk_model_revision="v2.0.2",
)
rec_result = funasr_model.generate(input=audio_input, batch_size_s=10, batch_size_threshold_s=10)
res_json = generate_json(rec_result[0]['sentence_info'])
del funasr_model
gc.collect()```