No response at random times

[uudruid74]

When I hit a button multiple times rapidly, and push a new command before the old one completes, I get multiple outputs. For example, if I hit a button 3 times very fast, I might get 6 commands sent rather than 3. Operator Error.

I'm currently having an issue where the server seems to not be responsive. Is there some way to debug this? I have to kill it and restart it constantly

[uudruid74]

Randomly getting this as well ... Traceback (most recent call last): File "/usr/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock self.process_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request self.finish_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python2.7/SocketServer.py", line 657, in init self.finish() File "/usr/lib/python2.7/SocketServer.py", line 716, in finish self.wfile.close() File "/usr/lib/python2.7/socket.py", line 279, in close self.flush() File "/usr/lib/python2.7/socket.py", line 303, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) error: [Errno 32] Broken pipe

[radinsky]

It looks to me like it can be a network issue..

[hackruu]

@uudruid74 try with older broadlink module: pip install -Iv broadlink==0.3

[uudruid74]

@radinsky - The network is working fine for everything else. And the web interface sending the AJAX request is running on the same physical server as the rest server. It's making a request to itself, so it shouldn't be a network problem.

@hackruu - Tried that and restarted the server, still no change. According to my waterfall diagram, the server takes about 20 seconds to respond for the initial request(s). Once it responds, it's fine (about 400ms response). Then it may randomly go to sleep again later. I'd say it was swapping but there is no swap configured (Beaglebone on Jessie).

[uudruid74]

I think the following is related ... https://stackoverflow.com/questions/10003866/http-server-hangs-while-accepting-packets

[uudruid74]

Adding timeout seems to help. the "nonblocking request" seems to block!

--- broadlink-http-rest-master/server.py    2017-08-10 10:30:09.000000000 -0500
+++ server.py   2018-03-22 18:23:15.353067594 -0500
@@ -8,6 +8,7 @@
 from os import path
 from Crypto.Cipher import AES

+address = "192.168.12.200"

 class Server(BaseHTTPRequestHandler):
     def _set_headers(self):
@@ -99,7 +100,7 @@
         finalCommand = encodedCommand[0x04:]

         signal.signal(signal.SIGALRM, signal_handler)
-        signal.alarm(4)  # Ten seconds
+        signal.alarm(2)  # Two seconds
         try:
             device.send_data(finalCommand)
         except Exception, msg:
@@ -178,9 +179,10 @@
     print ("HTTP timeout, but the command should be already sent.")

-def start(server_class=HTTPServer, handler_class=Server, port=serverPort):
-    server_address = ('', port)
+def start(server_class=HTTPServer, handler_class=Server, port=serverPort, timeout=1):
+    server_address = (address, port)
     httpd = server_class(server_address, handler_class)
+    httpd.timeout = timeout
     print 'Starting broadlink-rest server on port %s ...' % port
     httpd.serve_forever()

@@ -222,4 +224,4 @@
     else:
         serverPort = 8080

-    start(port=serverPort)
+    start(port=serverPort,timeout=RealTimeout)

[radinsky]

@uudruid74, can you commit your changes? I will merge your patch, thanks!

[uudruid74]

@radinksky False alarm. It's dead again, keyboard interrupt still shows it hanging in the same "non blocking" call. I'll keep looking into this and will do a proper fork/PR when I have a system that turns on my TV in the morning without restarting the server first 😃

[uudruid74]

I'm testing more changes, and this time, I think I have it. Basically setting a timeout in the device itself. I did a bunch of changes and this needs time to test. There may still be a slight delay when its been left alone for awhile, but I'm pretty sure thats just the hard drive on my Beaglebone spinning down. It doesn't hang like it used to. PR coming soon

[uudruid74]

@radinsky I issued a PR. Really confident this is finally dead and can be closed

False alarm. It's still there. Hanging in the socket code. I'm not sure if there is a solution other than avoiding socket.py and writing our own.

[uudruid74]

OK. I managed to find out how to replicate. When a client rudely disconnects it seems to hang the server. So, while I'm testing it, all is good. I hibernate my laptop and no one can connect the next morning! Even a page refresh of my client was enough to trigger.

I solved it via adding socket timeouts to the server process and then gracefully closing the connection when the error propogates. Timeouts are configurable as "Timeout" in the global section, in tenths of a second, with a fast default. PR coming soon.

[uudruid74]

Aspect of single-threaded design, resolved via timeouts in #13