jamesremuscat / avx

A system for controlling A/V devices such as video switchers via a networkable control device.
4 stars 1 forks source link

Unexpected client disconnects can block subsequent client calls #59

Closed jamesremuscat closed 8 years ago

jamesremuscat commented 8 years ago

Despite work on this area in the past, it's still possible for an unexpected disconnect of the client to cause issues on the server.

[2016-05-16 21:44:23.475,Pyro4.core,ERROR] cannot connect: [Errno 111] Connection refused
[2016-05-16 21:44:23.483,Pyro4.core,ERROR] problem getting metadata: cannot connect
[2016-05-16 21:44:23.491,root,ERROR] Failed to call function on registered client PYRO:obj_0e12dd3b47404ac09e1b59c72582edfa@192.168.1.121:51814, removing.
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/avx/controller/Controller.py", line 109, in callAllClients
    result = function(client)
  File "build/bdist.linux-x86_64/egg/avx/controller/Controller.py", line 176, in <lambda>
    self.callAllClients(lambda c: c.showPowerOffDialog())
  File "/home/pi/avx/local/lib/python2.7/site-packages/Pyro4-4.40-py2.7.egg/Pyro4/core.py", line 247, in __getattr__
    self._pyroGetMetadata()
  File "/home/pi/avx/local/lib/python2.7/site-packages/Pyro4-4.40-py2.7.egg/Pyro4/core.py", line 543, in _pyroGetMetadata
    self.__pyroCreateConnection()
  File "/home/pi/avx/local/lib/python2.7/site-packages/Pyro4-4.40-py2.7.egg/Pyro4/core.py", line 498, in __pyroCreateConnection
    raise ce
CommunicationError: cannot connect: [Errno 111] Connection refused
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "build/bdist.linux-x86_64/egg/avx/Sequencer.py", line 45, in run
    event.execute(self.controller)
  File "build/bdist.linux-x86_64/egg/avx/Sequencer.py", line 77, in execute
    getattr(controller, self.method)(*self.args)
  File "build/bdist.linux-x86_64/egg/avx/controller/Controller.py", line 176, in showPowerOffDialogOnClients
    self.callAllClients(lambda c: c.showPowerOffDialog())
  File "build/bdist.linux-x86_64/egg/avx/controller/Controller.py", line 113, in callAllClients
    self.clients.pop(uri)
TypeError: an integer is required

And repeated disconnects cause the number of clients we think we have to be excessive (actual max was two):

[2016-05-21 21:37:45.472,root,INFO] Unregistered client at PYRO:obj_e4a4b9fda7234d4984d8750981578f1f@192.168.1.103:53245
[2016-05-21 21:37:45.479,root,INFO] 8 client(s) still connected
[2016-05-21 21:38:09.274,root,INFO] Registered client at PYRO:obj_ffc1f63550a944ecb7ad4d0927ab4065@192.168.1.103:35480
[2016-05-21 21:38:09.281,root,INFO] 9 client(s) now connected
[2016-05-21 21:38:20.339,root,INFO] Unregistered client at PYRO:obj_ffc1f63550a944ecb7ad4d0927ab4065@192.168.1.103:35480
[2016-05-21 21:38:20.346,root,INFO] 8 client(s) still connected
[2016-05-21 21:59:22.223,root,INFO] Registered client at PYRO:obj_7be6bbf6572547309348042dc60c7089@192.168.1.103:59006
[2016-05-21 21:59:22.241,root,INFO] 9 client(s) now connected
[2016-05-21 22:01:41.491,root,INFO] Unregistered client at PYRO:obj_7be6bbf6572547309348042dc60c7089@192.168.1.103:59006
[2016-05-21 22:01:41.497,root,INFO] 8 client(s) still connected

Net result is that anything involving a call to callAllClients (e.g. Aldates system power!) will fail.

Until #49 is done, we should do better than this.