OWR to OWR broken

[superdump]

test-send-receive is very shaky, giving output which is very strange for localhost usage, like:

0:00:13.797182000  6613 0x7fb9b30d19e0 WARN         rtpjitterbuffer gstrtpjitterbuffer.c:2456:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer1> Packet #10423 too late as #10464 was already popped, dropping
0:00:13.797211000  6613 0x7fb9b30d19e0 WARN         rtpjitterbuffer gstrtpjitterbuffer.c:2456:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer1> Packet #10445 too late as #10464 was already popped, dropping
0:00:14.844525000  6613 0x7fb9b315b8f0 WARN               rtpsource rtpsource.c:1412:rtp_source_get_new_sr: no clock-rate, cannot interpolate rtp time
0:00:20.128171000  6613 0x7fb9b315b8f0 WARN               rtpsource rtpsource.c:1412:rtp_source_get_new_sr: no clock-rate, cannot interpolate rtp time
0:00:22.832046000  6613 0x7fb9b30d19e0 WARN         rtpjitterbuffer gstrtpjitterbuffer.c:2456:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer1> Packet #12999 too late as #13028 was already popped, dropping
0:00:22.832146000  6613 0x7fb9b30d19e0 WARN         rtpjitterbuffer gstrtpjitterbuffer.c:2456:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer1> Packet #13019 too late as #13028 was already popped, dropping
0:00:22.985626000  6613 0x7fb9b30d19e0 WARN         rtpjitterbuffer gstrtpjitterbuffer.c:2456:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer1> Packet #13068 too late as #13068 was already popped, dropping
0:00:25.350223000  6613 0x7fb9b315b8f0 WARN               rtpsource rtpsource.c:1412:rtp_source_get_new_sr: no clock-rate, cannot interpolate rtp time

Similarly using openwebrtc-daemon to video chat with itself gives blackness. I wonder if this is related to #299.

[sdroege]

Those RTP problems really shouldn't happen locally and hint at bigger problems somewhere else. I would first debug those and then after they are fixed it makes sense to look at anything at remains.

It might be related to the work I started on only starting to send packets once we actually can... because if we send too early stuff is even lost on localhost, and we also get bursts (that are currently not handled well in rtpjitterbuffer in GStreamer, there's a bug about that with an implementation plan for that)

[stefanalund]

I'm pretty sure I had NativeDemo working between two iPhones (using the older framework). Will need to double check.

[superdump]

244 seems relevant.

[stefanalund]

Rob, did you try NativeDemo on two iOS devices?

[stefanalund]

Why closing?

[superdump]

I suspect it was an accident. Reopening.

[superdump]

@stefanalund - no but the daemon and test-send-receive showed similar problems and @ijsf observed the same in #244 with iOS I think.

[ijsf]

Perhaps this commit makes a difference (see my relevant comment in #244): https://github.com/ijsf/openwebrtc/commit/997ac7e5b5a07572de2d6cd554d4788448c75d34

[superdump]

@ijsf - can you please submit pull requests for such changes?

[superdump]

I had iOS to safari and Safari to safari working the other day I think. But it was hit and miss. Sometimes it worked better than others. Sometimes just black. But it's clear we have some issues to address.

[ford-prefect]

(sorry about the close, I have no idea how that happened, maybe a misclick)

[stefanalund]

Latest NativeDemo works between 2 iPhones:

As pointed out before, this app currently uses an older version of the OpenWebRTC.framework.

[superdump]

@stefanalund - cool. This is maybe two issues or just one but the timing with the daemon is such that it triggers more severely.

test-send-receive also works, but exhibits intermittent black frames in the remote view quite a lot. Do you see bursts of black frames in the remote view with native demo against itself?

I'll investigate this and ijsf's patch more thoroughly on Tuesday.

[stefanalund]

Yes, I see long sequences of black frames on both sides occasionally. Pretty sure it's not a NW issue as I'm sitting very close to the AP.

When the video is good, it's really good, like FaceTime good 😀

[superdump]

test-send-receive is now working with VP8 after https://github.com/EricssonResearch/cerbero/commit/4ca6e8b48e177fc56a6c5a2f4769678be99fc523 which will get fixed properly in this issue https://bugzilla.gnome.org/show_bug.cgi?id=747208 .

However, with H264 I think it is still not working in test-send-receive. I will check again and respond here.

[superdump]

It seems that the H.264 not working between OWR and OWR is something going wrong in the h264parse code: https://gist.github.com/superdump/1cc1d2206ac648004ddb

It gets stuck in a loop of requesting a new keyframe and then deciding that it can't parse the NALU:

0:00:01.063659000 70742 0x7f882483c000 DEBUG      codecparsers_h264 gsth264parser.c:1419:gst_h264_parser_identify_nalu_avc: Can't parse, buffer has too small size 82929, offset 82929

Backtracing in that error case gives: https://gist.github.com/superdump/a779c7c6877ce7ed840f

It seems a bit strange to me that the offset (third argument) passed into gst_h264_parser_identify_nalu_avc in this case is offset + size and not just offset, but perhaps that's that's correct...? http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst/videoparsers/gsth264parse.c#n807 (the call is on line 865.)

I'm going to look att #299 some more and come back to this one.

[ford-prefect]

Switching VP8 to H264 in test-send-receive works for me here on Linux. Anything specific I need to do to reproduce what you're seeing?

The video is quite laggy, and out of sync with audio, but that's something to take up separately imo (possibly related to #186).

[superdump]

Agreed about sync and whatnot.

It could be something that changed with VideoToolbox then. There was a recent OS X and Xcode upgrade just a few days ago but I thought I saw this before then. Perhaps VideoToolbox is outputting a slightly different bitstream format now. Can you paste a gist of GST_DEBUG=h264:6? Maybe that will give clues.

[ford-prefect]

Here we go: http://arunraghavan.net/temp/log.h264.txt (it's got the color information in, so you'll need to use less -RS)

[superdump]

A small update - I have tested with disabling RTX in test-send-receive and then modifying owr_payload.c to either explicitly create vtenc_h264 or openh264enc on OS X. With openh264enc it works. With vtenc_h264 I see the issue I mentioned in the comment above: https://github.com/EricssonResearch/openwebrtc/issues/311#issuecomment-91482542 .

[superdump]

This seems to only happen with H264 in test-send-receive on my MacBook Pro using VideoToolbox for encoding. Removing from release milestone.