search

blacklanternsecurity / bbot

A recursive internet scanner for hackers.
https://www.blacklanternsecurity.com/bbot/
GNU General Public License v3.0
4.42k stars 394 forks source link

HTTP URLs Missed When Running through Burp #35

Open TheTechromancer opened 1 year ago

TheTechromancer commented 1 year ago

When BBOT is set to proxy through Burpsuite, it seems that httpx discovers HTTPS URLs but not HTTP ones.

liquidsec commented 1 year ago

Having trouble replicating, do you have an example query?

liquidsec commented 1 year ago

On hold until issue can be replicated

TheTechromancer commented 1 year ago

Unable to replicate, closing.

liquidsec commented 9 months ago

Was able to replicate this, there seems to be two separate, but related issues:

1) False reporting of port 80 on https sites: https://www.blacklanternsecurity.com:80/ 2) HTTP sites not being detected at all when running through burp

TheTechromancer commented 5 months ago

Running some tests this week, and starting to realize how bad this is. Httpx consistently misses URLs any time the web server issues a valid HTTP response on an HTTPS port (which practically every modern server and WAF does).

When given a target, it will output idiotic results:

[URL]                   http://aliexpress.com/  httpx   (dir, http-title-301-moved-permanently, in-scope, ip-47-246-173-237, status-301)
[URL]                   http://aliexpress.com:443/  httpx   (dir, http-title-301-moved-permanently, in-scope, ip-47-246-173-237, status-301)

http://aliexpress.com:443/?? There is no excuse for this. We should prioritize replacing httpx.

@liquidsec