Open Kogia-sima opened 7 months ago
I have the same problem, although I'm not even doing anything particularly heavy in my loop.
while ctx.state.playing:
with lock:
is_speaking = info["is_speaking"]
audio_frame_id = info["audio_frame_id"]
I even get delay if the only thing in the while loop is:
while ctx.state.playing:
pass
The lines above cause a delay of multiple seconds in the video delivered via webrtc:
def video_frame_callback(frame): return frame
My environment is:
OS: macOS Ventura 13.4.1 (arm) Shell: zsh Browser: Firefox 122 Python: 3.10.13 streamlit: 1.31.1 streamlit-webrtc: 0.47.1
I think it's because Python doesn't execute CPU-bound tasks in parallel even in multiple threads due to the GIL.
Off-loading the cpu-heavy tasks to another "process" by using multiprocessing
might be a solution.
Summary
CPU-heavy task inside the while-loop causes video frame delay within the streamlit-webrtc container.
minimal reproduction:
Details
I'm building a real-time image recognition application using streamlit-webrtc. However, the video in the application is extremely laggy and delayed. I examined this and found that the above minimal code causes same behavior. I also found that this issue has gone after I inserted manual sleep inside the loop. However, I can't figure out how long sleep is enough to avoid frame delay issue.
Question
Environment