Syncplay / syncplay

Client/server to synchronize media playback on mpv/VLC/MPC-HC/MPC-BE on many computers
http://syncplay.pl/
Apache License 2.0
2.11k stars 214 forks source link

Connection with server lost loop due to bittorrent traffic reaching Syncplay port #256

Closed zehric closed 4 years ago

zehric commented 4 years ago

I'm running syncplay server on linux with TLS enabled running on master branch. At some point, Windows clients running 1.6.4a and other lower versions started getting into this disconnect loop that looks like this:

[21:34:51] Attempting to connect to [REDACTED]:8999
[21:34:52] Attempting secure connection
[21:34:52] Successfully reached [REDACTED] ([REDACTED])
[21:34:52] Connection with server lost, attempting to reconnect
[21:34:52] Attempting secure connection
[21:34:52] Connection with server lost, attempting to reconnect
[21:34:52] Attempting secure connection
[21:34:53] Connection with server lost, attempting to reconnect
[21:34:53] Attempting secure connection
[21:34:53] Connection with server lost, attempting to reconnect
...

This only starts happening an indeterminate amount of time after starting up the server (less than 24 hours).

Looking at syncplay server's stdout, augmented by printing out the non-utf-8 string referenced by the error message:

https://gist.github.com/zehric/fba67e7deb8eb2da27e9f45a70024a11

Any idea what's going on here? This is difficult to debug because the issue only starts happening some long period of time after starting up the server, and I'm not even sure if the client drop error has anything to do with the connection loop.

albertosottile commented 4 years ago

Hi, thanks for using Syncplay. From your gist, line 6 b'\x13BitTorrent protocol\x00\x00\x00\x00 [...] it appears that something else is interfering with port 8999, either in the server or in the clients.

I'd advise you to check what is running on that port on your systems and to switch to another port number. If you are running your own server, you can pick basically any free port.

zehric commented 4 years ago

It turns out that the port number was indeed the problem. Since 8999 is the default port and a lot of torrent clients use that port, they seem to somehow be interfering with Syncplay. I feel like this issue should be kept active though, since Syncplay should be made to be more resistant to other apps sending extraneous data to it, otherwise DoS can happen pretty easily, whether intentional or not.

Et0h commented 4 years ago

I think the relevant line of code is https://github.com/Syncplay/syncplay/blame/master/syncplay/protocols.py#L44 self.dropWithError(getMessage("line-decode-server-error"))

Does anyone want to comment on if they think this line should be removed? And, if it is removed, should it give a notification to the server that it detected this error?

daniel-123 commented 4 years ago

After digging a bit it seems that once you connect to modern torrent network your IP/port gets passed around for a good while. This would perfectly explain observed behavior. Though there are few things of note to consider:

That said I think it's either a very large volume of garbage data (it normally can handle over 100 concurrently connected clients even on very low end hardware and with no restarts for weeks), or the server is doing something very wrong when dropping clients like this (like leaking crazy amounts of memory, spawning threads without ever closing them etc.). Latter scenario seems likely given how it only occurs after server runs for some time.

@zehric - what was the ballpark number of dropped clients in the stdout? Even just order of magnitude should be useful information. Would it be possible for you to replicate the issue, but with wireshark monitoring the traffic to port 8999? That could help confirming on denying the theory about it being torrents.

zehric commented 4 years ago

Some additional info that might be useful is that sometimes waiting long enough does result in clients being able to connect. I'd say ballpark around 50, sometimes less and sometimes more, connection retries would happen before the client successfully connected. Though it hasn't happened since I changed the port so I'm going off of memory here.

After a day of running, I'd say I probably saw less than 50 dropped clients in the stdout.

This issue isn't a huge priority of mine to solve since it was resolved by just changing the port number, so I'm not too keen on trying to replicate the issue for now. If it comes up again I'll look into it.

Et0h commented 4 years ago

I've added a link to this issue from https://syncplay.pl/guide/trouble/ so that others can see how to resolve this issue if it occurs for them. Given that this does not seem to be a commonly occurring problem I will be closing this issue. Thanks for your assistance in helping us better understand this issue and how to resolve it.

themegabyte commented 3 years ago

I don't wanna open this issue, but how would I enable the "debug mode" that @zehric posted above in a gist?

Appreciate the help.