search

jitsi / jitsi-videobridge

Jitsi Videobridge is a WebRTC compatible video router or SFU that lets build highly scalable video conferencing infrastructure (i.e., up to hundreds of conferences per server).
https://jitsi.org/jitsi-videobridge
Apache License 2.0
2.91k stars 989 forks source link

VideoBridge is crushing on Firefox 57 connection #588

Open vektory79 opened 6 years ago

vektory79 commented 6 years ago

I trying to connect to self hosted jitsi meet server. Audio is working perfect, but video doesn't. In vb.log I found these entry:

JVB 2017-12-21 12:54:42.178 WARNING: [34] org.jitsi.videobridge.xmpp.ComponentImpl.processIQ() (serving component 'JitsiVideobridge') Unexpected exception while processing IQ stanza: <iq type="set" to="jitsi-videobridge.vlasenko.krista.ru" from="focus@auth.vlasenko.krista.ru/focus17808996575009" id="4OC0a-2640"><conference xmlns="http://jitsi.org/protocol/colibri" id="c4ff62de641be7bb" name="testroom" gid="fff12e"><content name="audio"><channel id="a055d7bbff830d70"><source xmlns="urn:xmpp:jingle:apps:rtp:ssma:0" ssrc="3561494476"><parameter value="af9c432d-bf4d-4a03-b48a-aa2df67940cd" name="cname"/><parameter value="fa3a6a2d-2fb2-42cf-852f-500540b70251 53122841-d662-4c0b-b9ca-87fa42c4f0ab" name="msid"/><ssrc-info xmlns="http://jitsi.org/jitmeet" owner="testroom@conference.vlasenko.krista.ru/8543bb17"/></source><source xmlns="urn:xmpp:jingle:apps:rtp:ssma:0" ssrc="165604091"><parameter value="af9c432d-bf4d-4a03-b48a-aa2df67940cd" name="cname"/><ssrc-info xmlns="http://jitsi.org/jitmeet" owner="testroom@conference.vlasenko.krista.ru/8543bb17"/></source></channel></content><content name="video"><channel id="d29a344cf50e499c"><source xmlns="urn:xmpp:jingle:apps:rtp:ssma:0" ssrc="3210773065"><parameter value="af9c432d-bf4d-4a03-b48a-aa2df67940cd" name="cname"/><parameter value="3462df3b-bb9d-4a0c-9095-c7358ae146f1 c2b2b6dd-e451-47c2-831d-9895197873ee" name="msid"/><ssrc-info xmlns="http://jitsi.org/jitmeet" owner="testroom@conference.vlasenko.krista.ru/8543bb17"/></source><ssrc-group xmlns="urn:xmpp:jingle:apps:rtp:ssma:0" semantics="SIM"/></channel></content></conference></iq>
java.lang.ArrayIndexOutOfBoundsException: 0
        at org.jitsi.impl.neomedia.rtp.MediaStreamTrackReceiver.setMediaStreamTracks(MediaStreamTrackReceiver.java:177)
        at org.jitsi.videobridge.RtpChannel.setRtpEncodingParameters(RtpChannel.java:1932)
        at org.jitsi.videobridge.VideoChannel.setRtpEncodingParameters(VideoChannel.java:289)
        at org.jitsi.videobridge.Videobridge.handleColibriConferenceIQ(Videobridge.java:926)
        at org.jitsi.videobridge.Videobridge.handleColibriConferenceIQ(Videobridge.java:612)
        at org.jitsi.videobridge.xmpp.ComponentImpl.handleIQRequest(ComponentImpl.java:433)
        at org.jitsi.videobridge.xmpp.ComponentImpl.handleIQ(ComponentImpl.java:347)
        at org.jitsi.videobridge.xmpp.ComponentImpl.handleIQ(ComponentImpl.java:299)
        at org.jitsi.videobridge.xmpp.ComponentImpl.handleIQSetImpl(ComponentImpl.java:546)
        at org.jitsi.xmpp.component.ComponentBase.handleIQSet(ComponentBase.java:362)
        at org.xmpp.component.AbstractComponent.processIQRequest(AbstractComponent.java:515)
        at org.xmpp.component.AbstractComponent.processIQ(AbstractComponent.java:289)
        at org.xmpp.component.AbstractComponent.processQueuedPacket(AbstractComponent.java:239)
        at org.xmpp.component.AbstractComponent.access$100(AbstractComponent.java:81)
        at org.xmpp.component.AbstractComponent$PacketProcessor.run(AbstractComponent.java:1051)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

As I understand, the root cause is the incorrect IQ packet from Firefox:

<iq type="set" to="jitsi-videobridge.vlasenko.krista.ru" from="focus@auth.vlasenko.krista.ru/focus17808996575009" id="4OC0a-2640">
    <conference xmlns="http://jitsi.org/protocol/colibri" id="c4ff62de641be7bb" name="testroom" gid="fff12e">
        <content name="audio">
            <channel id="a055d7bbff830d70">
                <source xmlns="urn:xmpp:jingle:apps:rtp:ssma:0" ssrc="3561494476">
                    <parameter value="af9c432d-bf4d-4a03-b48a-aa2df67940cd" name="cname"/>
                    <parameter value="fa3a6a2d-2fb2-42cf-852f-500540b70251 53122841-d662-4c0b-b9ca-87fa42c4f0ab" name="msid"/>
                    <ssrc-info xmlns="http://jitsi.org/jitmeet" owner="testroom@conference.vlasenko.krista.ru/8543bb17"/>
                </source>
                <source xmlns="urn:xmpp:jingle:apps:rtp:ssma:0" ssrc="165604091">
                    <parameter value="af9c432d-bf4d-4a03-b48a-aa2df67940cd" name="cname"/>
                    <ssrc-info xmlns="http://jitsi.org/jitmeet" owner="testroom@conference.vlasenko.krista.ru/8543bb17"/>
                </source>
            </channel>
        </content>
        <content name="video">
            <channel id="d29a344cf50e499c">
                <source xmlns="urn:xmpp:jingle:apps:rtp:ssma:0" ssrc="3210773065">
                    <parameter value="af9c432d-bf4d-4a03-b48a-aa2df67940cd" name="cname"/>
                    <parameter value="3462df3b-bb9d-4a0c-9095-c7358ae146f1 c2b2b6dd-e451-47c2-831d-9895197873ee" name="msid"/>
                    <ssrc-info xmlns="http://jitsi.org/jitmeet" owner="testroom@conference.vlasenko.krista.ru/8543bb17"/>
                </source>
                <ssrc-group xmlns="urn:xmpp:jingle:apps:rtp:ssma:0" semantics="SIM"/>
            </channel>
        </content>
    </conference>
</iq>

in which ssrc-group is empty.

Is this the bug in the Firefox or in Video Bridge?

bbaldino commented 6 years ago

this looks like the issue that was fixed here: https://github.com/jitsi/jitsi-videobridge/commit/4722a5bfb3e83539683643c6557cd23a148078b5

vektory79 commented 6 years ago

Sadly it's seems like that commit is introduce this issue. How it's looks for me:

  1. Firefox sends IQ with empty source group
  2. that fix replace null source group with empty list
  3. downstream code try to access hardcoded first element in this empty list
  4. as a result we have an ArrayIndexOutOfBoundsException by index 0

Do I missed something?

bbaldino commented 6 years ago

Oops, you're right. Sorry, didn't look closely enough at the actual exception. So are the sources in your example also empty? An empty groups list wouldn't prevent tracks from being created, I'd think, but maybe there's a detail in there I'm missing. Either way, assuming there's one track doesn't seem great.

As far as the format of the source-group (being in-line to the element as opposed to nested), I don't remember what's supported there off the top of my head. Is this with the experimental firefox setting enabled?

gpolitis commented 6 years ago

I'm able to repro the issue and it's currently blocking @damencho's work on the tests. I can have a look unless you want to pinch in @bbaldino. I'm debating the idea of rewriting the factory method the proper way.

bbaldino commented 6 years ago

I'm still confused as to why sources are empty, even though there is a source in the colibri. Is this due to the code being confused about a sim group but only a single source?

I'm also still confused as to how this is even happening on firefox. @vektory79 is this with the enableFirefoxSimulcast setting enabled? Without it, we shouldn't even be doing simulcast on firefox.

vektory79 commented 6 years ago

Sorry, but I away from test cluster and work computer till Monday. But it's very likely, that I activate enableFirefoxSimulcast in the past and forgot for that :-/

Do I understand properly, that removing this settings can normalize the situation?

gpolitis commented 6 years ago

@vektory79 what happens if you downgrade your bridge to a previously known version? are you able to bake your own bridge? we have a proposed fix for this issue here https://github.com/jitsi/jitsi-videobridge/pull/589

vektory79 commented 6 years ago

@bbaldino parameter enableFirefoxSimulcast is not enabled in my config.

@gpolitis sadly I can't compile this commit:

[ERROR] /media/data/Java/jitsee/jitsi-videobridge/src/main/java/org/jitsi/videobridge/xmpp/MediaStreamTrackFactory.java:[211,21] constructor MediaStreamTrackDesc in class org.jitsi.impl.neomedia.rtp.MediaStreamTrackDesc cannot be applied to given types;
  required: org.jitsi.impl.neomedia.rtp.MediaStreamTrackReceiver,org.jitsi.impl.neomedia.rtp.RTPEncodingDesc[],boolean
  found: org.jitsi.impl.neomedia.rtp.MediaStreamTrackReceiver,boolean
  reason: actual and formal argument lists differ in length
[ERROR] /media/data/Java/jitsee/jitsi-videobridge/src/main/java/org/jitsi/videobridge/xmpp/MediaStreamTrackFactory.java:[212,18] cannot find symbol
  symbol:   method setRTPEncodings(org.jitsi.impl.neomedia.rtp.RTPEncodingDesc)
  location: variable track of type org.jitsi.impl.neomedia.rtp.MediaStreamTrackDesc
[ERROR] /media/data/Java/jitsee/jitsi-videobridge/src/main/java/org/jitsi/videobridge/xmpp/MediaStreamTrackFactory.java:[223,15] constructor MediaStreamTrackDesc in class org.jitsi.impl.neomedia.rtp.MediaStreamTrackDesc cannot be applied to given types;
  required: org.jitsi.impl.neomedia.rtp.MediaStreamTrackReceiver,org.jitsi.impl.neomedia.rtp.RTPEncodingDesc[],boolean
  found: org.jitsi.impl.neomedia.rtp.MediaStreamTrackReceiver,boolean
  reason: actual and formal argument lists differ in length
[ERROR] /media/data/Java/jitsee/jitsi-videobridge/src/main/java/org/jitsi/videobridge/xmpp/MediaStreamTrackFactory.java:[244,25] incompatible types: possible lossy conversion from long to int
[ERROR] /media/data/Java/jitsee/jitsi-videobridge/src/main/java/org/jitsi/videobridge/xmpp/MediaStreamTrackFactory.java:[261,14] cannot find symbol
  symbol:   method setRTPEncodings(org.jitsi.impl.neomedia.rtp.RTPEncodingDesc[])
  location: variable track of type org.jitsi.impl.neomedia.rtp.MediaStreamTrackDesc

Will try to use older version prior to 4722a5b

bbaldino commented 6 years ago

@vektory79 if the firefox simulcast param was not enabled then this seems very weird. can you enable the peer connection trace logging, repro the issue, and get the logs from firefox? something is going wrong to end up with that sdp.

To enable the trace logs, you'll want to uncomment the block of code here: https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/TraceablePeerConnection.js#L216

bbaldino commented 6 years ago

@gpolitis i will take a look at those PRs soon but i think we need to understand what's even causing this weird sdp before we fix the NPE issue...then it may just be masking something else going wrong that will manifest in some other way.

bbaldino commented 6 years ago

@vektory79 i haven't been able to reproduce this on meet.jit.si or beta.meet.jit.si (though we have seen another issue with firefox video not getting forwarded, but not a crash). are you still seeing it? are you on tip of master?