opus not working with chan_pjsip and Asterisk 13.1

[goseeped]

Test for Webrtc peer Opus and VP8

OS: ubuntu 14.04 Asterisk: 13.1.0 current version with this patch for 13.1 , it has slin16 audios Client : jssip 0.6.12 online demo ( disable new session timers feature ) /firefox 34 channel : Chan_sip

Test 1) Call between two webrtc peers firefox 34 jssip client playback audio before dial.

On Playback(letters/asterisk); works great On Dial ; i only get this warnings

WARNING[10405][C-00000001]: res_srtp.c:407 ast_srtp_unprotect: SRTP unprotect failed with: authentication failure 10

[Jan 28 17:37:00] WARNING[10405][C-00000001]: res_srtp.c:407 ast_srtp_unprotect: SRTP unprotect failed with: authentication failure 110

Results

• Audio opus works great.
• Video VP8 not work , on ast 11 this patch it work with same clients and config,.

Asterisk in this case is sending media to both legs of the call but Firefox Video debug said "Received incomplete frame timestamp" and "Decoder error: -1"

DEBUG ; (11:42:16:178 | 0) VIDEO CODING: 0 1; 6376; Packet seqnum=42311 timestamp=14902920 inserted at 10216406 DEBUG ; (11:42:16:179 | 1) VIDEO CODING: 0 1; 6506; Delay: min_playout=0 jitter=150 max_decode=0 render=10 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; ExtrapolateLocalTime(14809680)=10215464 ms DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Render frame 3159780769 at 14809680. Render delay 10215624 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Delay: min_playout=0 jitter=150 max_decode=0 render=10 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Received incomplete frame timestamp 14801220 frame size 1492 at time 10215416 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Packet received and sent to jitter estimate with: timestamp=14801220 wall_clock=10215416 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Jitter estimate updated with: frameSize=1492 frameDelayMS=22 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Framesize statistics: max=2884.646530 average=1591.919289 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; The estimated slope is: theta=(0.007451, 31.791275) DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Random jitter: mean=-18.282323 variance=4748.316652 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Current jitter estimate: 140.188457 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Current max RTT: 0 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; g1=0.000000 g2=0.000000 alarm=0 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; w[0]=89.329760 w[1]=-363864.617750 ts=14809680 tMs=168322 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 1; 6506; Delay: min_playout=0 jitter=150 max_decode=0 render=10 DEBUG ; (11:42:16:179 | 0) VIDEO CODING: 0 0; 6506; Decoding timestamp 14809680 ERROR ; (11:42:16:179 | 0) VIDEO CODING: 0 0; 6506; Decoder error: -1 ERROR ; (11:42:16:179 | 0) VIDEO CODING: 0 0; 6506; Failed to decode frame 14809680, requesting key frame

Test 2) Curious thing is that if you call from linphone udp vp8 to web browser , linphone could see video from web browser but web browser couldn't see video from linphone, seems like asterisk is changing something for video for VP8 streams when webrtc peer is involve.

I will test on chan_pjsip and let you know, video VP8 over webrtc peers work for you on chan_sip or chan_pjsip?

Signalling seems to be OK , since i can't attach captures here becuase of format i could email you if you need it .

Thanks Sean for you effort and for this patch would be great have video working , let me know your thoughts.

[seanbright]

The patch that I have here for Asterisk 13 does not implement VP8 passthrough. Asterisk itself now has the VP8 passthrough support built-in. You should open an issue there: https://issues.asterisk.org/

[goseeped]

Oh i see , i think you have added some extra changes becuase i was able to pass video VP8 with linphone without this patch , but since you add a new format file on this patch i though you have some changes related to it !

[seanbright]

Nothing in this patch should affect VP8 passthrough. Do a fresh install of Asterisk 13.1, apply asterisk.patch, and copy the stuff in codecs to the codecs directory in the Asterisk distribution. Do not copy format_vp8.c in. If you continue to experience problems, let me know.

[goseeped]

Oh maybe i explain wrong , before the your patch and after the patch with ast 13.1 on chan_sip, i could make VP8 call with linphone's that's ok with patch or not , the thing that not work is video on WebRTC peers ! actually opus codecs work for WebRTC peers this on chan_sip. now i just test with "chan_pjsip" and opus audio not work asterisk decline opus offer. so i guess i have to post something on issue.asterisk.org about video thing :S . Thanks

[seanbright]

OK. So to make sure I understand. This opus patch does not work with chan_pjsip on Asterisk 13.1? Is that correct?

[goseeped]

This opus patch not work for chan_pjsip. Asterisk signalling response "SIP/2.0 488 Not Acceptable Here" when you put allow=opus on pjsip.conf .

so the answer is : not work for chan_pjsip.

Sorry for my english.

[seanbright]

Got it. Will take a look.

[goseeped]

Wow thanks for take a look ! anything i could help on test let me know. actually i have already open a ticket on asterisk lira for video part let's see what happen ! Regards

[raarts]

It tried it with asterisk 13.2.0-rc1, and does not work either. I dial a Yealink phone from Chrome/sipml5. I forced Chrome to opus (allow=opus) and the phone to g722. Got the following:

[Feb 2 23:52:09] WARNING[1743][C-00000007]: chan_pjsip.c:530 chan_pjsip_answer: Unable to push answer task to the threadpool. Cannot answer call [Feb 2 23:52:09] DEBUG[1743][C-00000007]: codec_opus.c:281 opustolin_destroy: Destroyed decoder #23 (opus->16000) [Feb 2 23:52:09] DEBUG[1743][C-00000007]: codec_opus.c:281 opustolin_destroy: Destroyed decoder #21 (opus->48000) [Feb 2 23:52:09] DEBUG[1743][C-00000007]: channel.c:2699 ast_hangup: Hanging up channel 'PJSIP/102-0000000f' [Feb 2 23:52:09] DEBUG[1743][C-00000007]: chan_pjsip.c:1644 hangup_cause2sip: AST hangup cause 16 (no match found in PJSIP) [Feb 2 23:52:09] DEBUG[1735]: res_pjsip_session.c:1912 handle_outgoing_request: Method is BYE [Feb 2 23:52:09] DEBUG[946]: res_config_mysql.c:1583 mysql_reconnect: MySQL RealTime: Connection okay. [Feb 2 23:52:09] DEBUG[946]: res_config_mysql.c:377 realtime_mysql: MySQL RealTime: Retrieve SQL: SELECT * FROM ps_endpoints WHERE id = '102' [Feb 2 23:52:09] DEBUG[946]: res_sorcery_realtime.c:119 sorcery_realtime_filter_objectset: Filtering out realtime field 'disallow' from retrieval [Feb 2 23:52:09] DEBUG[946]: devicestate.c:464 do_state_change: Changing state for PJSIP/102 - state 2 (In use) [Feb 2 23:52:09] DEBUG[1006]: app_queue.c:2374 device_state_cb: Device 'PJSIP/102' changed to state '2' (In use) but we don't care because they're not a member of any queue. [Feb 2 23:52:09] DEBUG[1743][C-00000007]: app_dial.c:3105 dial_exec_full: Exiting with DIALSTATUS=ANSWER. [Feb 2 23:52:09] DEBUG[1743][C-00000007]: pbx.c:6441 __ast_pbx_run: Spawn extension (default,102,1) exited non-zero on 'PJSIP/101-0000000e' == Spawn extension (default, 102, 1) exited non-zero on 'PJSIP/101-0000000e' [Feb 2 23:52:09] DEBUG[1743][C-00000007]: channel.c:2550 ast_softhangup_nolock: Soft-Hanging (0x10) up channel 'PJSIP/101-0000000e' [Feb 2 23:52:09] DEBUG[1743][C-00000007]: codec_opus.c:281 opustolin_destroy: Destroyed decoder #22 (opus->48000) [Feb 2 23:52:09] DEBUG[1743][C-00000007]: channel.c:2699 ast_hangup: Hanging up channel 'PJSIP/101-0000000e' [Feb 2 23:52:09] DEBUG[1736]: res_pjsip_session.c:1927 handle_outgoing_response: Method is INVITE, Response is 488 Not Acceptable Here <--- Transmitting SIP response (561 bytes) to WS:192.168.241.10:50116 ---> SIP/2.0 488 Not Acceptable Here

When I set both channels to g722 or ulaw, everything works.

[seanbright]

Does opus passthrough work with chan_pjsip for either of you?

[sam123sam123]

I can also confirm OPUS not working on chan_pjsip on Asterisk 13.2.0 and Asterisk 12.8.1 while doing an echo test or between 2 extensions using pjsip v2.3 library compiled from source tarball. OPUS works fine on chan_sip. I confirmed I was using the correct branch of asterisk-opus for each version of Asterisk.

Both extensions are behind NAT while the server is not. So I don't think that is working as passthrough.

[sam123sam123]

I confirmed passthrough does not work with chan_pjsip either with direct_media=yes and no NAT between extensions and server. Switched to chan_sip and it worked. There was no codec_opus installed on the Asterisk server for this test so I know it was using passthrough.

Here are the errors from the CLI

res_pjsip_sdp_rtp.c:247 set_caps: No joint capabilities for 'audio' media stream between our configuration((ulaw|alaw|gsm|g726|speex|opus)) and incoming SDP((nothing))

chan_pjsip.c:530 chan_pjsip_answer: Unable to push answer task to the threadpool. Cannot answer call

[mrm1734]

I also seem to be having trouble with Opus/PJSIP on Asterisk 13.3.2. I am running on an ARM Cortex A8, and everything else seems to be working great. This is my last little hurdle! I try to dial in to my little "hello world" test and get this from the CLI:

WARNING[14675][C-00000004]: chan_pjsip.c:534 chan_pjsip_answer: Unable to push answer task to the threadpool. Cannot answer call

I followed the instructions in the readme and the only "opus" command I have is "opus show" which gives the following:

*CLI> opus show 3/0 encoders/decoders are in use.

There doesn't appear to be any command for "opus set debug" but I do see OPUS listed when I run the "core show translation" command.

Has there been any breakthroughs on this lately?

[demon-ru]

Except that I can confirm all that is written here. I was very pleased to support the opus codec, but was upset that is not supported res_pjsip. Will follow the updates. Really looking forward to.

[traud]

This issue was followed-up in ASTERISK-24779. Until this is solved within Asterisk itself, a patch is attached to that issue report there, which enables the Opus Cidec within Asterisk 13 and chan_pjsip.

For those interested to debug such things: I followed the Wiki about debugging, especially core set debug 6 gave me .SDP negotiation done, status=220049 before chan_pjsip_answer: Unable to push answer task to the threadpool. Cannot answer call Then, I downloaded the source of PJSIP and searched for 220049, which is PJMEDIA_SDPNEG_NOANSCODEC. This is returned in pjmedia/sdp_neg.c:match_offer. Because the matching failed, I added PJ_LOG(1, ("", "%u:%s %u:%s %u:%s", or_.enc_name.slen, or_.enc_name.ptr, lr.enc_name.slen, lr.enc_name.ptr, lr.param.slen, lr.param.ptr)); below the comment See if encoding name, clock-rate, channel-count match. Conclusion: Opus was not registered correctly with its channel count. Therefore, PJSIP assumed a channel count of 1. Then, I had to find the place in Asterisk, where all codecs are attached to PJSIP. Then, I was lucky and found the patch from Sean.

[sam123sam123]

OPUS now works with PJSIP on Asterisk v13.7.0.

One problem I have with it is that it seems to negotiate each direction separately and defaults to ulaw or alaw for incoming to my SIP client even though I have OPUS as the top priority codec on both sides. If I disable all codecs on my SIP client except OPUS then I get OPUS in both directions. Unless it's a problem with my setup this seems to be a bug.

[traud]

Thanks for the update. Yes, Asterisk 13.7 solved this issue thanks to the patch from Sean. The issue was an upstream in Asterisk 13. Therefore here in this Opus patch project, no changes were required.

@shadowym because you created a separate report for this bi-directional negotiation issue – #17 does cover the same, doesn’t it? – I am closing this issue here.