Closed warownia1 closed 4 years ago
Use
loop.run_until_complete(asyncio.gather(*self.workers, self.server))
instead of
loop.run_forever()
to propagate all exceptions and cause the explicit error in the main thread
Surround asyncio.create_subprocess_shell
with try ... except
block, create master thread which manages and restarts interrupted workers or use semaphores and spawn a new worker for each job instead of having worker pool
Using the pool you don't use the main feature as asyncio where is shines -- little to no overhead when creating new threads. With asyncio you should spawn short-lived gthreads instead of reusing them. Additionally, having one worker per job makes cancelling/stopping easier.
When multiple jobs are submitted to the local queue and then deleted, a missing working directory causes FileNotFound exception when the process is spawned. The exception is silently ignored and the worker handling this job is stopped without warning.