search

signalwire / freeswitch

FreeSWITCH is a Software Defined Telecom Stack enabling the digital transformation from proprietary telecom switches to a versatile software implementation that runs on any commodity hardware. From a Raspberry PI to a multi-core server, FreeSWITCH can unlock the telecommunications potential of any device.
https://freeswitch.com/#getting-started
Other
3.52k stars 1.41k forks source link

Audio with OPUS codec is broken #1865

Open figaro2015 opened 1 year ago

figaro2015 commented 1 year ago

Describe the bug Audio with the OPUS codec is broken when added to a video-only call.

To Reproduce Steps to reproduce the behavior:

  1. Establish video-only call leg by placing a call from FreeSWITCH to the VoIP softphone far end supporting video (for example Linphone) using originate API and having outbound_codec_prefs global variable set to VP8 in the vars.xml. Example of the API invocation: originate sofia/external/1004@192.168.86.66 3000 XML public. The public dial plan instruction for 3000 could be placing the call into the conference.
  2. Add an audio component to the call by calling uuid_media_reneg API and specifying both video (VP8) and audio (OPUS) codecs as parameters. Example of the API invocation: uuid_media_reneg 4e71887c-981e-43f9-b3f6-fc3138a3d5e3 =VP8,OPUS. Important to note the OPUS has to be used as an audio codec.
  3. Observe that only some distortion and random clicking can be heard on the far end instead of clear audio. Clearly indicates failure in decoding the received audio packets.

Expected behavior Clear audio should be heard from the VoIP softphone client.

Package version or git hash FreeSWITCH Version 1.10.9-dev+git~20221107T233635Z~bb682fc5b4~64bit (git bb682fc 2022-11-07 23:36:35Z 64bit)

Trace logs Here is the console output which gets produced by the OPUS codec when opus_debug is turned on:

2022-11-17 10:17:35.955789 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [93] FEC[no] channels[1] 2022-11-17 10:17:35.955789 99.93% [DEBUG] mod_opus.c:507 decode: opus_frames [2] samples [480] audio bandwidth [NARROWBAND] bytes [49] FEC[no] channels[1] 2022-11-17 10:17:35.975790 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [98] FEC[no] channels[1] 2022-11-17 10:17:35.975790 99.93% [ERR] mod_opus.c:497 decode: OPUS_INVALID_PACKET ! frames: -4 2022-11-17 10:17:35.995789 99.93% [ERR] mod_opus.c:921 Decoder Error: corrupted stream fs:4080 plc:false! 2022-11-17 10:17:35.995789 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [96] FEC[no] channels[1] 2022-11-17 10:17:35.995789 99.93% [ERR] mod_opus.c:497 decode: OPUS_INVALID_PACKET ! frames: -4 2022-11-17 10:17:36.015799 99.93% [ERR] mod_opus.c:921 Decoder Error: corrupted stream fs:4080 plc:false! 2022-11-17 10:17:36.015799 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [106] FEC[no] channels[1] 2022-11-17 10:17:36.015799 99.93% [ERR] mod_opus.c:497 decode: OPUS_INVALID_PACKET ! frames: -4 2022-11-17 10:17:36.015799 99.93% [ERR] mod_opus.c:921 Decoder Error: corrupted stream fs:4080 plc:false! 2022-11-17 10:17:36.035805 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [104] FEC[no] channels[1] 2022-11-17 10:17:36.035805 99.93% [ERR] mod_opus.c:497 decode: OPUS_INVALID_PACKET ! frames: -4 2022-11-17 10:17:36.055799 99.93% [ERR] mod_opus.c:921 Decoder Error: corrupted stream fs:4080 plc:false! 2022-11-17 10:17:36.055799 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [101] FEC[no] channels[1] 2022-11-17 10:17:36.055799 99.93% [ERR] mod_opus.c:497 decode: OPUS_INVALID_PACKET ! frames: -4 2022-11-17 10:17:36.055799 99.93% [ERR] mod_opus.c:921 Decoder Error: corrupted stream fs:4080 plc:false! 2022-11-17 10:17:36.075799 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [100] FEC[no] channels[1] 2022-11-17 10:17:36.075799 99.93% [ERR] mod_opus.c:497 decode: OPUS_INVALID_PACKET ! frames: -4 2022-11-17 10:17:36.095789 99.93% [ERR] mod_opus.c:921 Decoder Error: corrupted stream fs:4080 plc:false! 2022-11-17 10:17:36.095789 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [97] FEC[no] channels[1] 2022-11-17 10:17:36.095789 99.93% [ERR] mod_opus.c:497 decode: OPUS_INVALID_PACKET ! frames: -4 2022-11-17 10:17:36.115788 99.93% [ERR] mod_opus.c:921 Decoder Error: corrupted stream fs:4080 plc:false! 2022-11-17 10:17:36.115788 99.93% [DEBUG] mod_opus.c:507 encode: opus_frames [1] samples [960] audio bandwidth [FULLBAND] bytes [96] FEC[no] channels[1]

Analysis Audio with OPUS codec works perfectly fine when both audio and video media components are present when the initial call is established. Something gets configured wrong only when the audio component is getting added only after the video-only call is established. Important to note that video in all scenarios is perfectly fine. By adding some debugging I can confirm that OPUS codec in all cases is getting configured identically. Based on that, looks like it is reasonable to suspect some issue with either buffers allocation or timing of the audio packets.

dragos-oancea commented 1 year ago

can we have the full debug log of the call ?

figaro2015 commented 1 year ago

call.log

figaro2015 commented 1 year ago

Please see the full debug log with the SIP trace turned on attached. Thanks!

dragos-oancea commented 1 year ago

I see the leg is in conference, does it happen without conference too (eg: answer, record_session and playback something) ? does it happen with jitter buffer enabled ? Is it the same if the reinvite is sent from Linphone ?

figaro2015 commented 1 year ago

Yes to all questions. The quintessential scenario which causes the problem to be exposed is OPUS codec audio being added to the existing video-only calls. When both media components (video and audio) are present at the initial call establishment - everything works fine. Also, everything seems to be working fine with any other audio codec (PCMU, PCMA, G722).