wnielson / Plex-Remote-Transcoder

A distributed transcoding backend for Plex
MIT License
646 stars 58 forks source link

Speed up Host selection #21

Open DrEVILish opened 8 years ago

DrEVILish commented 8 years ago

Get each servers load python in async mode instead of waiting for each to respond, timeout after 1 seconds.

If one of the hosts is offline then ignore it. Speeds up the percieved response time, also if one slave has responded in (y) time. do Y*2 to be the total wait time for the rest of the servers. If they don't reply within the given time then pick the least busy this should speed up load times of media firing when a PRT_Slave is offline.

wnielson commented 7 years ago

I looked in to this a bit today and significant performance gains can be made without any modification to the code. SSH provides a way to create persistent connections that can be multiplexed. This is best illustrated with a simple example.

Let's say we have a transcode slave with address 10.99.0.11. On the master, as the plex user, create an SSH config file nano ~/.ssh/config and add the following:

Host 10.99.0.11
        HostName 10.99.0.11
        ControlPath ~/.ssh/controlmasters/%r@%h:%p
        ControlMaster auto
        ControlPersist 2h

Make sure to create the directory where SSH will store the sockets: mkdir ~/.ssh/controlmasters. Now, test it out:

plex@prt-1:~$ time ssh 10.99.0.11 hostname && prt get_load
prt-2

real    0m0.164s
user    0m0.036s
sys 0m0.001s
3.0 6.0 5.0

The first time we run this it takes 164ms. Let's run it again:

plex@prt-1:~$ time ssh 10.99.0.11 hostname && prt get_load
prt-2

real    0m0.028s
user    0m0.008s
sys 0m0.010s
1.0 5.0 5.0

This time it took only 28ms--that's over 5 times faster, cool.

At any time, we can check to see if the connection is alive:

plex@prt-1:~$ ssh -O check 10.99.0.11
Master running (pid=31022)

We can also manually close the connection:

plex@prt-1:~$ ssh -O stop 10.99.0.11
Stop listening request sent.

This should just work with PRT--if the connection doesn't exist, SSH will create it and if it does exist it will be used. In my extremely limited testing, this seems to work just fine with PRT.