Open Jackiexiao opened 3 years ago
Hi, please try below.
The reason is that your app script is re-executed every time component state (e.g. playing or not) changes then audio_buffer
object differs in the execution where the audio recording loop is running and the next execution where .export()
is called.
This re-execution is Streamlit's fundamental execution model (See https://docs.streamlit.io/en/stable/main_concepts.html#app-model),
and it provides some ways to handle problems like this case, and the example below uses st.session_state
to keep the same sound object over executions.
import queue
import pydub
import streamlit as st
from streamlit_webrtc import webrtc_streamer, WebRtcMode, ClientSettings
def main():
webrtc_ctx = webrtc_streamer(
key="sendonly-audio",
mode=WebRtcMode.SENDONLY,
audio_receiver_size=256,
client_settings=ClientSettings(
rtc_configuration={
"iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}]
},
media_stream_constraints={
"audio": True,
},
),
)
if "audio_buffer" not in st.session_state:
st.session_state["audio_buffer"] = pydub.AudioSegment.empty()
status_indicator = st.empty()
while True:
if webrtc_ctx.audio_receiver:
try:
audio_frames = webrtc_ctx.audio_receiver.get_frames(timeout=1)
except queue.Empty:
status_indicator.write("No frame arrived.")
continue
status_indicator.write("Running. Say something!")
sound_chunk = pydub.AudioSegment.empty()
for audio_frame in audio_frames:
sound = pydub.AudioSegment(
data=audio_frame.to_ndarray().tobytes(),
sample_width=audio_frame.format.bytes,
frame_rate=audio_frame.sample_rate,
channels=len(audio_frame.layout.channels),
)
sound_chunk += sound
if len(sound_chunk) > 0:
st.session_state["audio_buffer"] += sound_chunk
else:
status_indicator.write("AudioReciver is not set. Abort.")
break
audio_buffer = st.session_state["audio_buffer"]
if not webrtc_ctx.state.playing and len(audio_buffer) > 0:
st.info("Writing wav to disk")
audio_buffer.export("temp.wav", format="wav")
# Reset
st.session_state["audio_buffer"] = pydub.AudioSegment.empty()
if __name__ == "__main__":
main()
If your purpose is only to record the sound to a file without any processing with pydub
, recorder option is the simplest way:
from streamlit_webrtc import webrtc_streamer, WebRtcMode, ClientSettings
from aiortc.contrib.media import MediaRecorder
def main():
def recorder_factory():
return MediaRecorder("record.wav")
webrtc_streamer(
key="sendonly-audio",
mode=WebRtcMode.SENDONLY,
in_recorder_factory=recorder_factory,
client_settings=ClientSettings(
rtc_configuration={
"iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}]
},
media_stream_constraints={
"audio": True,
"video": False,
},
),
)
if __name__ == "__main__":
main()
thx very much for your help, I will dive into webRTC and streamlit for details, thxs
If your purpose is only to record the sound to a file without any processing with
pydub
, recorder option is the simplest way:from streamlit_webrtc import webrtc_streamer, WebRtcMode, ClientSettings from aiortc.contrib.media import MediaRecorder def main(): def recorder_factory(): return MediaRecorder("record.wav") webrtc_streamer( key="sendonly-audio", mode=WebRtcMode.SENDONLY, in_recorder_factory=recorder_factory, client_settings=ClientSettings( rtc_configuration={ "iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}] }, media_stream_constraints={ "audio": True, "video": False, }, ), ) if __name__ == "__main__": main()
I use sample code as above, but it doesn't work as expected if you play it (seems it missing half of audio frame), it used to work before.
if you put wavfile in audition, you get ( by the way, record wav using webrtc_ctx.audio_receiver.get_frames(timeout=1)
works fine
@whitphx
mm, while I'm not familiar with sound processing and cannot understand what the figures mean,
I wonder if setting some options to the MediaRecorder
works.
MediaRecorder
passes the options
parameter to FFmpeg (via the underlying PyAV
library). So, what if you set options like options={"sample_rate": ..., "channels": ...}
?
set options doesn't work, but if I change mode to mode=WebRtcMode.SENDRECV
instead of SENDONLY
, MediaRecorder works fine
I write a gist here for reproduce
@whitphx @Jackiexiao I just did in your @Jackiexiao's gist. I played it right after I recorded it, and it seems the speed of the .wav
file is two times faster. How could I solve this problem?
@hihunjin use this function to record https://gist.github.com/Jackiexiao/9c45d5ad9caf524471b5ed966c57b5b9#file-streamlit_recorder-py-L131
@Jackiexiao thank you for your investigation and problem-solving.
The fact that SENDRECV
instead SENDONLY
works fine is interesting hint. Thanks!
Hi! Im using the code that you post there https://gist.github.com/Jackiexiao/9c45d5ad9caf524471b5ed966c57b5b9#file-streamlit_recorder-py-L131. When i click the START button it works just fine but when i stop it the audio file doesn't save it anywhere. I don't know if a need to change some parameter or something like that. Sorry im new coding with streamlit :)
Actually im getting this log https://github.com/whitphx/streamlit-webrtc/issues/552 Any idea on how to solve it :/?
Hi @whitphx and @Jackiexiao, this code is very helpful. Thanks a lot! I wonder how to, by default, turn off the speaker while recording. The goal is to prevent audio feedback when not wearing headphones.
You can control the HTML attributes of the <audio />
element through the audio_html_attrs
of webrtc_streamer()
component as below.
webrtc_ctx: WebRtcStreamerContext = webrtc_streamer(
key="sendonly-audio",
mode=WebRtcMode.SENDRECV,
in_recorder_factory=recorder_factory,
media_stream_constraints=MEDIA_STREAM_CONSTRAINTS,
audio_html_attrs={"muted": True} # THIS!
)
When you set only muted=true
, the audio component is completely hidden. So if you want to show the audio control and mute the audio at the same time, set controls=true
too as below.
audio_html_attrs={"muted": True, "controls": True}
You can find other available attributes: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/audio
I want to use streamlit-webrtc to record wav and save to disk (like
temp.wav'), but I have some problem, after I click
stopbutton, audio_buffer become
None` instead of audio from microphonemy code