peaclab / MicroFaaS-worker

FaaS on small, embedded-system-like compute nodes
MIT License
0 stars 1 forks source link

Unnecessary warnings/exceptions when shutting down VMWorkers #13

Closed abyrne55 closed 3 years ago

abyrne55 commented 3 years ago

When a VMWorker transitions into the OFF state, sometimes the pkill command fails to halt the corresponding VM before it makes another worker request. This triggers the following warning/exception:

Aug 19 13:36:57 beaglebone python3[20799]: WARNING:root:VMWorker104 made request but no output events set

and/or

Aug 19 13:36:57 beaglebone python3[20799]: ----------------------------------------
Aug 19 13:36:57 beaglebone python3[20799]: Exception happened during processing of request from ('192.168.1.104', 42042)
Aug 19 13:36:57 beaglebone python3[20799]: Traceback (most recent call last):
Aug 19 13:36:57 beaglebone python3[20799]:   File "/usr/lib/python3.7/socketserver.py", line 650, in process_request_thread
Aug 19 13:36:57 beaglebone python3[20799]:     self.finish_request(request, client_address)
Aug 19 13:36:57 beaglebone python3[20799]:   File "/usr/lib/python3.7/socketserver.py", line 360, in finish_request
Aug 19 13:36:57 beaglebone python3[20799]:     self.RequestHandlerClass(request, client_address, self)
Aug 19 13:36:57 beaglebone python3[20799]:   File "/usr/lib/python3.7/socketserver.py", line 720, in __init__
Aug 19 13:36:57 beaglebone python3[20799]:     self.handle()
Aug 19 13:36:57 beaglebone python3[20799]:   File "/home/debian/MicroFaaS/orchestrator.py", line 103, in handle
Aug 19 13:36:57 beaglebone python3[20799]:     self.data = self.request.recv(12288).strip()
Aug 19 13:36:57 beaglebone python3[20799]: ConnectionResetError: [Errno 104] Connection reset by peer
Aug 19 13:36:57 beaglebone python3[20799]: ----------------------------------------

These warnings/exceptions can be safely caught and ignored, as pkill ensures the VM is properly shut down, even if it lets a few erroneous worker requests slip out during the second or two it needs to kill QEMU.

(Note that this is in reference to code that's currently on the refactor branch.)

abyrne55 commented 3 years ago

Hopefully fixed by 57edd69