Refactor ssh_scan_worker bin/lib

mozilla / ssh_scan_api

An API for ssh_scan (https://github.com/mozilla/ssh_scan) and the backend API service for the Mozilla SSH Observatory (https://observatory.mozilla.org/)

31 stars 8 forks source link

Refactor ssh_scan_worker bin/lib #22

Open claudijd opened 7 years ago

claudijd commented 7 years ago

The workers are not as robust as I would prefer. Things that I think we could do better include...

Having a worker manager (that can spin up multiple worker processes in a single command-line run)
Have some sort of service recovery (that can make sure if a worker dies/dead-locks that it is respawned)
Have some sort of system service (that can ensure service start back up on a reboot)

claudijd commented 7 years ago

I'd prefer to have both thread and process control defined in the worker config so we can scan at both the thread and process level.

claudijd commented 7 years ago

We also need full-stop way of trapping exceptions on workers so they never get dead-ended on an unexpected stack trace and are restarted. This is effectively a bare rescue of sorts, but we want a way to report this somewhere so the information isn't lost and we can see the highest hitting bugs to go fix.