Issue on elevenlabs_synthesizer Decoding failed

[melvlt]

Hello,

First, thanks for this great project.

When we try to play a prompt after updating to the latest version we get the following error : 'pydub.exceptions.CouldntDecodeError:Decoding failed. ffmpeg returned error code: 1'.

Can you please help us with this ?

Regards

`2024-10-21 16:26:17.613 INFO {task_manager} [_synthesize] ##### sending text to elevenlabs for generation: Salut Melvyn, comment ça va ? 2024-10-21 16:26:17.613 INFO {task_manager} [_synthesize] Cached response and hence sending preprocessed text 2024-10-21 16:26:17.614 INFO {task_manager} [send_preprocessed_audio] Time to get response from S3 3.027869388461113e-05 2024-10-21 16:26:17.614 INFO {task_manager} [__send_preprocessed_audio] File doesn't exist in S3. Hence we're synthesizing it from synthesizer 2024-10-21 16:26:17.614 INFO {task_manager} [_synthesize] ##### sending text to elevenlabs for generation: Salut Melvyn, comment ça va ? 2024-10-21 16:26:17.614 INFO {elevenlabs_synthesizer} [push] Pushed message to internal queue {'data': 'Salut Melvyn, comment ça va ?', 'meta_info': {'io': 'twilio', 'message_category': 'agent_welcome_message', 'stream_sid': 'stream_51001', 'request_id': '82a64b71-a75d-4776-bad7-d681723bca25', 'cached': False, 'sequence_id': -1, 'format': 'pcm', 'text': 'Salut Melvyn, comment ça va ?', 'is_md5_hash': False, 'llm_generated': False, 'type': 'audio', 'synthesizer_start_time': 1729520777.6145315}} 2024-10-21 16:26:17.614 INFO {elevenlabs_synthesizer} [push] end_of_llm_stream: False 2024-10-21 16:26:17.617 INFO {elevenlabs_synthesizer} [sender] Waiting for elevenlabs ws connection to be established... 2024-10-21 16:26:17.740 INFO {elevenlabs_synthesizer} [establish_connection] Connected to wss://api.elevenlabs.io/v1/text-to-speech/VgLkEnMc5gZbC7iaZyjz/stream-input?model_id=eleven_turbo_v2_5&output_format=ulaw_8000&inactivity_timeout=60 2024-10-21 16:26:18.618 INFO {elevenlabs_synthesizer} [sender] Sending text_chunk: Salut 2024-10-21 16:26:18.618 INFO {elevenlabs_synthesizer} [sender] Sending text_chunk: Melvyn, 2024-10-21 16:26:18.618 INFO {elevenlabs_synthesizer} [sender] Sending text_chunk: 2024-10-21 16:26:18.618 INFO {elevenlabs_synthesizer} [sender] Sending text_chunk: comment 2024-10-21 16:26:18.618 INFO {elevenlabs_synthesizer} [sender] Sending text_chunk: ça 2024-10-21 16:26:18.618 INFO {elevenlabs_synthesizer} [sender] Sending text_chunk: va 2024-10-21 16:26:18.619 INFO {elevenlabs_synthesizer} [sender] Sending text_chunk: ? 2024-10-21 16:26:19.543 INFO {task_manager} [check_for_completion] Last transmitted timestamp is simply 0 and hence continuing 2024-10-21 16:26:21.543 INFO {task_manager} [__check_for_completion] Last transmitted timestamp is simply 0 and hence continuing 2024-10-21 16:26:22.543 INFO {elevenlabs_synthesizer} [receiver] response for isFinal: None 2024-10-21 16:26:22.543 INFO {elevenlabs_synthesizer} [generate] Received message from server 2024-10-21 16:26:22.543 INFO {utils} [convert_audio_to_wav] CONVERTING AUDIO TO WAV mp3 Traceback (most recent call last): File "/opt/poc_bolna/bolna/synthesizer/elevenlabs_synthesizer.py", line 165, in generate audio = resample(convert_audio_to_wav(message, source_format="mp3"), int(self.sampling_rate), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/poc_bolna/bolna/helpers/utils.py", line 356, in convert_audio_to_wav audio = AudioSegment.from_file(io.BytesIO(audio_bytes), format=source_format) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/poc_bolna/.venv/lib/python3.11/site-packages/pydub/audio_segment.py", line 773, in from_file raise CouldntDecodeError( pydub.exceptions.CouldntDecodeError: Decoding failed. ffmpeg returned error code: 1

Output from ffmpeg/avlib:

ffmpeg version 5.1.6-0+deb12u1 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 12 (Debian 12.2.0-14) configuration: --prefix=/usr --extra-version=0+deb12u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared libavutil 57. 28.100 / 57. 28.100 libavcodec 59. 37.100 / 59. 37.100 libavformat 59. 27.100 / 59. 27.100 libavdevice 59. 7.100 / 59. 7.100 libavfilter 8. 44.100 / 8. 44.100 libswscale 6. 7.100 / 6. 7.100 libswresample 4. 7.100 / 4. 7.100 libpostproc 56. 6.100 / 56. 6.100 [cache @ 0x56254a7a0680] Inner protocol failed to seekback end : -38 Last message repeated 1 times [mp3 @ 0x56254a79fe40] Failed to read frame size: Could not seek to 12521. [cache @ 0x56254a7a0680] Statistics, cache hits:2 cache misses:1 cache:pipe:0: Invalid argument

2024-10-21 16:26:22.705 INFO {elevenlabs_synthesizer} [generate] Error in eleven labs generate Decoding failed. ffmpeg returned error code: 1

Output from ffmpeg/avlib:

ffmpeg version 5.1.6-0+deb12u1 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 12 (Debian 12.2.0-14) configuration: --prefix=/usr --extra-version=0+deb12u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared libavutil 57. 28.100 / 57. 28.100 libavcodec 59. 37.100 / 59. 37.100 libavformat 59. 27.100 / 59. 27.100 libavdevice 59. 7.100 / 59. 7.100 libavfilter 8. 44.100 / 8. 44.100 libswscale 6. 7.100 / 6. 7.100 libswresample 4. 7.100 / 4. 7.100 libpostproc 56. 6.100 / 56. 6.100 [cache @ 0x56254a7a0680] Inner protocol failed to seekback end : -38 Last message repeated 1 times [mp3 @ 0x56254a79fe40] Failed to read frame size: Could not seek to 12521. [cache @ 0x56254a7a0680] Statistics, cache hits:2 cache misses:1 cache:pipe:0: Invalid argument`

[slnw]

same here!

[juanantoroz]

i've also been having this issue since the 0.9.0 version