mozilla / ssh_scan_api

An API for ssh_scan (https://github.com/mozilla/ssh_scan) and the backend API service for the Mozilla SSH Observatory (https://observatory.mozilla.org/)
31 stars 8 forks source link

Add queue caching #31

Closed claudijd closed 7 years ago

claudijd commented 7 years ago

This is a stop gap to prevent queue spamming and the eventual result of hitting the same target over and over in a very short period of time.

It helps mitigate #27 in the queue, but not after the queue item is consumed by a worker. This will be completely resolved once we get to queue status (queued, running, completed) being maintained in the DB, which is partially underway in https://github.com/mozilla/ssh_scan_api/pull/2

This also swaps out the pre-existing Ruby Queue implementation with a Set-based queue to make it slightly more usable for managing and checking for pre-existing queue duplicates.