search

collabora / WhisperLive

A nearly-live implementation of OpenAI's Whisper.
MIT License
2.1k stars 286 forks source link

fix: limit CPU usage for VAD onnxruntime inference session by setting… #215

Closed makaveli10 closed 6 months ago

makaveli10 commented 6 months ago

This PR introduces an enhancement to control the number of threads used by OpenMP via the OMP_NUM_THREADS environment variable, which by default is set to 1 and can also be controlled via a command-line argument.

Screenshot from 2024-05-24 14-39-12

ONNX Runtime and OpenMP onnxruntime uses OpenMP for parallelism to improve performance on multi-core systems. By setting OMP_NUM_THREADS, we can control the number of threads used during the execution of ONNX models, ensuring efficient resource utilization and predictable performance.

Fixes https://github.com/collabora/WhisperLive/issues/201