cedricroijakkers commented 4 years ago

Description

When using a Linux client, small changes in any network interface cause an immediate disconnection from a meeting room

Current behavior

Connect to a meeting room as usual. When nothing special happens, the connection remains open for hours.
Execute a specific command that changes something small in the network configuration of the client. So far, we've had 3 distinct people reproduce it the following way: a) build or start a docker container, b) start a vagrant virtual machine (using VirtualBox), c) connect or disconnect a VPN
You will be immediately through out of the meeting room, reconnect after a few seconds, where you can still see your previous connection, after a minute or so the previous connection will get dropped
You can repeat this process, and you will be disconnected again

Expected Behavior

Do not get disconnected when executing one of these commands. I'm running a private Jitsi instance using the docker containers, this behaviour has only started appearing since version stable-4548-1, previously this did not happen. Nothing else changed in the OS.

Possible Solution

None yet

Steps to reproduce

Connect to a Jitsi room, using version stable-4548-1
Build a docker container/start a docker container/start a VirtualBox vm/connect or disconnect a VPN (tested with a Fortinet SSLVPN)

Environment details

Clients: Running Ubuntu 20.04 LTS, not sure if this is relevant, but this happens to be the distro that all people that have this problem are running here
Jitsi server: Docker containers version stable-4548-1

I've managed to reproduce this both in the browser (Google Chrome) as in the Jitsi Meet Electron app. Doesn't seem to be client related, it started happening since I updated the server.

saghul commented 4 years ago

Can you reproduce the behavior when connecting to meet.jit.si ?

cedricroijakkers commented 4 years ago

Just tried it on meet.jit.si, and it doesn't seem to happen there. There is one slight difference though, because of the way I'm running Jitsi in Azure Kubernetes, connections over UDP don't work (I blame Microsoft for this). So when having 3 or more people in a meeting room, we always switch back to TCP which works fine for the rest.

I will see if I can grab some logs when this happens.

cedricroijakkers commented 4 years ago

In the browser console I get the following:

Logger.js:154 2020-05-19T13:15:44.644Z [modules/xmpp/xmpp.js] <A.connectionHandler>:  (TIME) Strophe connfail[item-not-found]:   103194.38000000082
Logger.js:154 2020-05-19T13:15:44.645Z [modules/xmpp/xmpp.js] <A.connectionHandler>:  (TIME) Strophe disconnected[item-not-found]:   103195.22499999948
Logger.js:154 2020-05-19T13:15:44.645Z [modules/xmpp/strophe.ping.js] <p.stopInterval>:  Ping interval cleared
Logger.js:154 2020-05-19T13:15:44.646Z [modules/statistics/statistics.js] <Function.S.sendAnalyticsAndLog>:  {"type":"operational","action":"connection.failed","attributes":{"error_type":"connection.otherError","error_message":"item-not-found","suspend_time":0,"time_since_last_success":null}}
Logger.js:154 2020-05-19T13:15:44.650Z [features/overlay] <Object.componentDidMount>:  The conference will be reloaded after 30 seconds.
Logger.js:154 2020-05-19T13:15:44.654Z [features/base/redux] <Object._listener>:  TypeError: Object(...)(...).getId is not a function
    at Object.listener (middleware.js:234)

[... stacktrace]

Logger.js:154 2020-05-19T13:15:44.665Z [modules/UI/videolayout/VideoLayout.js] <Object.removeParticipantContainer>:  Removing remote video: bfea8b5d
Logger.js:154 2020-05-19T13:15:44.665Z [modules/UI/videolayout/SmallVideo.js] <y.remove>:  Remove thumbnail bfea8b5d
Logger.js:154 2020-05-19T13:15:44.671Z [modules/UI/videolayout/VideoLayout.js] <Object.removeParticipantContainer>:  Removing remote video: e9915900
Logger.js:154 2020-05-19T13:15:44.672Z [modules/UI/videolayout/SmallVideo.js] <y.remove>:  Remove thumbnail e9915900
Logger.js:154 2020-05-19T13:15:44.680Z [modules/UI/videolayout/SmallVideo.js] <g.remove>:  Remove thumbnail 63aa9152
Logger.js:154 2020-05-19T13:15:44.683Z [modules/rttmonitor/rttmonitor.js] <h.stop>:  Stopping RttMonitor.
Logger.js:154 2020-05-19T13:15:44.684Z [modules/e2eping/e2eping.js] <h.stop>:  Stopping e2eping
Logger.js:154 2020-05-19T13:15:44.686Z [modules/xmpp/ChatRoom.js] <_.doLeave>:  do leave hilko45socialeisolatie@muc.meet.jitsi/63aa9152
Logger.js:154 2020-05-19T13:15:44.692Z [modules/UI/videolayout/VideoLayout.js] <Object.removeParticipantContainer>:  No remote video for bfea8b5d
o @ app.bundle.min.js?v=4074:126
removeParticipantContainer @ tslib.es6.js:17

[... stacktrace]

Logger.js:154 2020-05-19T13:15:44.693Z [conference.js] <a.<anonymous>>:  USER bfea8b5d LEFT: u {_jid: "hilko45socialeisolatie@muc.meet.jitsi/bfea8b5d", _id: "bfea8b5d", _conference: se, _displayName: "Vernier", _supportsDTMF: false, …}
Logger.js:154 2020-05-19T13:15:44.694Z [JitsiConference.js] <se._maybeStartOrStopP2P>:  Will start P2P with: hilko45socialeisolatie@muc.meet.jitsi/e9915900 after 5 seconds...
Logger.js:154 2020-05-19T13:15:44.697Z [modules/UI/videolayout/VideoLayout.js] <Object.removeParticipantContainer>:  No remote video for e9915900
o @ app.bundle.min.js?v=4074:126

[... stacktrace]

Logger.js:154 2020-05-19T13:15:44.698Z [conference.js] <a.<anonymous>>:  USER e9915900 LEFT: u {_jid: "hilko45socialeisolatie@muc.meet.jitsi/e9915900", _id: "e9915900", _conference: se, _displayName: "Hilko", _supportsDTMF: false, …}
Logger.js:154 2020-05-19T13:15:44.699Z [JitsiConference.js] <se._maybeClearDeferredStartP2P>:  Cleared deferred start P2P task
Logger.js:154 2020-05-19T13:15:44.704Z [modules/RTC/TraceablePeerConnection.js] <A.close>:  Closing TPC[2,p2p:false]...
content-script.js:24 content script: event received
content-script.js:25 MessageEvent
Logger.js:154 2020-05-19T13:15:44.723Z [JitsiMeetJS.js] <Object.getGlobalOnErrorHandler>:  UnhandledError: null Script: null Line: null Column: null StackTrace:  Error: Not connected
    at d.send (XmppConnection.js:398)
    at _.doLeave (ChatRoom.js:261)
    at ChatRoom.js:1501
    at new Promise (<anonymous>)
    at _.leave (ChatRoom.js:1478)
    at se.leave (JitsiConference.js:598)
    at a.ee (conference.js:428)
    at a.emit (events.js:151)
    at A.connectionHandler (xmpp.js:305)
    at d._stropheConnectionCb (XmppConnection.js:255)
    at I.Connection._changeConnectStatus (strophe.umd.js:3011)
    at I.Connection._doDisconnect (strophe.umd.js:3052)
    at s._interceptDoDisconnect (strophe.stream-management.js:207)
    at I.Connection._dataRecv (strophe.umd.js:3140)
    at D.Bosh._onRequestStateChange (strophe.umd.js:5012)

Logger.js:154 2020-05-19T13:15:44.725Z [JitsiConferenceEventManager.js] <a.<anonymous>>:  Ignored ENDPOINT_MESSAGE_RECEIVED for not existing participant: bfea8b5d {id: 11, type: "e2e-ping-response"}
s @ md5.js:1
(anonymous) @ md5.js:1
a.emit @ lib-jitsi-meet.min.js?v=4074:1
e.onmessage @ md5.js:1
Logger.js:154 2020-05-19T13:15:44.765Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onerror>:  Channel error: undefined
s @ md5.js:1

[... stacktrace]

Logger.js:154 2020-05-19T13:15:44.766Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onclose>:  Channel closed by client
Logger.js:154 2020-05-19T13:15:44.975Z [modules/UI/videolayout/LargeVideoManager.js] hover in local
Logger.js:154 2020-05-19T13:15:46.651Z [features/base/storage] <Object.persistState>:  redux state persisted. 0a1c9dc22ffb823603b71324935d7c94 -> 0f818c68f1d386d14a0d8489d98ac862

And on the JVB server I get a lot of these:

May 19, 2020 3:10:33 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Failed to receive: java.net.SocketException: Failed to receive data from socket.
May 19, 2020 3:10:33 PM org.jitsi.utils.logging2.LoggerImpl log
WARNING: Removing the active socket. Won't be able to send until a new one is elected.
May 19, 2020 3:10:33 PM org.jitsi.utils.logging2.LoggerImpl log
SEVERE: Error sending packet
java.io.IOException: No active socket.
        at org.ice4j.socket.MergingDatagramSocket.send(MergingDatagramSocket.java:202)
        at org.ice4j.socket.DelegatingDatagramSocket.send(DelegatingDatagramSocket.java:776)
        at org.jitsi.videobridge.transport.ice.IceTransport.send(IceTransport.kt:222)
        at org.jitsi.videobridge.Endpoint.doSendSrtp(Endpoint.java:470)
        at org.jitsi.nlj.util.PacketInfoQueue$sam$org_jitsi_utils_queue_PacketQueue_PacketHandler$0.handlePacket(PacketInfoQueue.kt)
        at org.jitsi.utils.queue.PacketQueue$HandlerAdapter.handleItem(PacketQueue.java:575)
        at org.jitsi.utils.queue.AsyncQueueHandler$1.run(AsyncQueueHandler.java:141)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Seems like the connection just dies?

saghul commented 4 years ago

If you are using TCP (which is a terrible idea btw) then yeah, if it's cut it's cut. When media flows over UDP it acn recover if there are gaps. The connection sshould be reestablisshed, but I think the media connection dies sooner in your case.

I cannot recommend deploying Jitsi on a place where UDP works strong enough.

cedricroijakkers commented 4 years ago

I know, but until Azure gets its load balancers fixed, TCP is all we have now. Technically speaking, according to their documentation they should send the UDP packets to the containers, and they seem to be configured that way, but for some reason Jitsi is not picking them up.

What's interesting is why the connection is being cut when I simply build a docker container on my laptop, and this only happens in the latest version of JItsi, not before. I know this is not much to go on, but remarkable none the less.

saghul commented 4 years ago

Moving to the docker project.

richardrey commented 4 years ago

Exactly the same thing happens to me I have two environments for my part pre-production and production to previously check the correct operation of the application.

jitse in the last update causes problems in the meeting rooms, closing the session and indicating:

What I have been able to make clear is that in the version:

jitsi / jvb fd8201652975 jitsi / jicofo 773ffe263260 jitsi / prosody 5734ef478ff3 jitsi / web 7afd18fc710a

this problem does not pass.

but in the current versions, it does not allow its correct operation, same configuration, same environment actalizing only the image causes those faults could you please help us with a possible solution.

current versions that don't work for me:

jitsi / jvb ff366d6653e3 jitsi / jicofo 8a99d30ab075 jitsi / prosody 50f5c1e719aa jitsi / web 7de9a63e7e82

saghul commented 4 years ago

@richardrey How do you know it’s the same problem? Can you share your browser console logs?

dieug commented 4 years ago

and i leave a mark here. absolutely the same problem. 1 user connects\disconnects cisco vpn while being in conf and all this conf destroys with video/audio artefacts (broken pieces of video and sounds). strongly confirmed for macos and vpn on/off. (i think, with linux/bsd all be the same).

appears difference with users quantity: 1-4 users in conf: disconnects only user with onoffing vpn. 20-30 users in conf: all conference falls around.

debian 10.3 jicofo 1.0-549-1 jitsi-meet 2.0.4468-1 jitsi-meet-prosody 1.0.4025-1 jitsi-meet-web 1.0.4025-1 jitsi-meet-web-config 1.0.4025-1 jitsi-videobridge2 2.1-183-gdbddd169-1 server: 8cpu/16Gb Ram, VM from mail.ru cloud (don't know a vm technology. full vm, not chroot or docker or openvz or etc).

it'll be worth, if i thinking right: this happens not only to one conference, it touches another on same server.

jitsi / docker-jitsi-meet

Linux clients: network interface changes cause disconnection from meeting room #612

Description

Current behavior

Expected Behavior

Possible Solution

Steps to reproduce

Environment details