blacklanternsecurity / bbot

A recursive internet scanner for hackers.
https://www.blacklanternsecurity.com/bbot/
GNU General Public License v3.0
4.02k stars 370 forks source link

Optimize Status Message #1423

Closed TheTechromancer closed 1 month ago

TheTechromancer commented 1 month ago

This PR simplifies the third status message as requested in https://github.com/blacklanternsecurity/bbot/issues/1422.

The status message normally looks like this:

[INFO] adorable_rodney: 478,746 events in queue (DNS_NAME: 425,699, IP_ADDRESS: 46,825, DNS_NAME_UNRESOLVED: 4,214, OPEN_TCP_PORT: 862, URL_UNVERIFIED: 710, HTTP_RESPONSE: 220, URL: 179, SOCIAL: 16, STORAGE_BUCKET: 15, TECHNOLOGY: 5, AZURE_TENANT: 1)

And has been changed to this:

[INFO] adorable_rodney: 478,746 events in queue

The reason was because the process of iterating through all the module queues and counting the event types was taking a very long time, roughly 10 seconds per 500K events. This could easily double or triple the length of the scan.

If this feature is requested to be brought back, we'll want to implement a custom queue with a stats counter that keeps state.

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 85.71429% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 92%. Comparing base (2901e26) to head (16cfd28).

Files Patch % Lines
bbot/scanner/manager.py 86% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## dev #1423 +/- ## ===================================== + Coverage 92% 92% +1% ===================================== Files 321 321 Lines 20685 20676 -9 ===================================== + Hits 19002 19014 +12 + Misses 1683 1662 -21 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.