Gowitness Didn't Run

[amiremami]

Gowitness module didn't run and didn't produce any screenshots.

debug.log output.json

[amiremami]

It seems because of using proxy, gowitness is not running.

[TheTechromancer]

The issue appears to be with your proxy:

This indicates the proxy isn't accepting connections.

[amiremami]

Hey @TheTechromancer thanks a lot, they said they fixed it, I don't get anymore connection refused in debug.log , however, still don't get any screenshots, is this still a proxy issue?

debug.log output.json

[TheTechromancer]

Hmm, that's strange. I see the URLs in there at least. Can you run the scan with -d?

[amiremami]

Here you are:

debug.log output.json

[TheTechromancer]

2024-04-16 17:35:24,834 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("https://myaccounting.it/", module=httpx, tags={'dir', 'in-scope', 'status-301'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:26,532 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("http://myaccounting.it/", module=httpx, tags={'dir', 'in-scope', 'status-301'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:35,441 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("http://www.myaccounting.it/", module=httpx, tags={'dir', 'in-scope', 'status-301'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:35,442 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("http://mx.myaccounting.it/", module=httpx, tags={'status-302', 'http-title-302-found', 'in-scope', 'dir'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:44,460 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("http://load.gtm.myaccounting.it/", module=httpx, tags={'dir', 'in-scope', 'http-title-301-moved-permanently', 'status-301'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:48,500 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("https://www.areaclienti.myaccounting.it/", module=httpx, tags={'status-302', 'in-scope', 'dir'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:48,501 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("https://areaclienti.myaccounting.it/", module=httpx, tags={'dir', 'in-scope', 'status-301'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:49,807 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("http://www.areaclienti.myaccounting.it/", module=httpx, tags={'dir', 'in-scope', 'status-301'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:49,955 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("http://areaclienti.myaccounting.it/", module=httpx, tags={'dir', 'in-scope', 'status-301'}) because it did not meet custom filter criteria: URL is a redirect
2024-04-16 17:35:59,747 [DEBUG] bbot.modules.gowitness base.py:1214 Not accepting URL("https://www.areaclienti.myaccounting.it/area-clienti/", module=httpx, tags={'dir', 'in-scope', 'status-301'}) because it did not meet custom filter criteria: URL is a redirect

Based on this it looks like due to the proxy, httpx is missing some of the https URLs. This is probably not the proxy's fault; this is a known issue with httpx. We have a very old bug open for this: https://github.com/blacklanternsecurity/bbot/issues/35.

We really need to replace this tool with something decent.

It's hard to tell but there may also be an issue with redirections. There are some pretty long redirect chains here, like http://areaclienti.myaccounting.it/ --> https://areaclienti.myaccounting.it/ --> https://www.areaclienti.myaccounting.it/ --> https://www.areaclienti.myaccounting.it/area-clienti/login/?redirect=https%3A%2F%2Fwww.areaclienti.myaccounting.it%2F.

[TheTechromancer]

The following URLs did pass post-check, so they were processed by gowitness. It's unclear why there were no screenshots for them:

2024-04-16 17:35:28,511 [DEBUG] bbot.modules.gowitness base.py:1214 URL("https://www.myaccounting.it/", module=httpx, tags={'status-200', 'dir', 'in-scope', 'http-title-myaccounting-it-studio-di-cont'}) passed post-check
2024-04-16 17:35:42,264 [DEBUG] bbot.modules.gowitness base.py:1214 URL("https://load.gtm.myaccounting.it/", module=httpx, tags={'status-400', 'in-scope', 'dir'}) passed post-check
2024-04-16 17:35:45,597 [DEBUG] bbot.modules.gowitness base.py:1214 URL("https://gtm.myaccounting.it:80/", module=httpx, tags={'dir', 'in-scope', 'status-404'}) passed post-check
2024-04-16 17:35:47,052 [DEBUG] bbot.modules.gowitness base.py:1214 URL("https://gtm.myaccounting.it/", module=httpx, tags={'status-400', 'in-scope', 'dir'}) passed post-check
2024-04-16 17:35:53,645 [DEBUG] bbot.modules.gowitness base.py:1214 URL("https://www.areaclienti.myaccounting.it/area-clienti/login/", module=httpx, tags={'in-scope', 'http-title-login-myaccounting-it', 'status-200', 'login-page', 'dir'}) passed post-check

I'd recommend running gowitness manually to see if it spits out any errors:

/root/.bbot/tools/gowitness --chrome-path /root/.bbot/tools/chrome-linux/chrome --db-path /root/.bbot/scans/cheeky_snape/gowitness/gowitness.sqlite3 --screenshot-path /root/.bbot/scans/cheeky_snape/gowitness/screenshots --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.2151.97' --proxy socks5://14ac7cb2f8d2c:4e90f0e479@91.190.191.94:12324 --resolution-x 1440 --resolution-y 900 file -f - --threads 4

[amiremami]

Thanks. I don't think it's because of redirections, Because it's not possible to get screenshots from any site.

Nothing printed here:

[TheTechromancer]

You need to pipe the urls into it.

[amiremami]

Sorry,

[TheTechromancer]

Seems to be another issue with the proxy. It might be worth trying a basic curl to verify a basic web request works through the proxy.

[amiremami]

I used these commands and it seems works fine:

curl -x socks5://14ac7cb2f8d2c:4e90f0e479@91.190.191.94:12324 https://www.myaccounting.it/
curl -x socks5://14ac7cb2f8d2c:4e90f0e479@91.190.191.94:12324 davcrkdidfhlhgvabwxp2nmjt0mkbpti9.oast.fun

[TheTechromancer]

Ah okay. Apparently the issue is that chromium doesn't support socks5 auth: https://github.com/puppeteer/puppeteer/issues/1074

[amiremami]

Thanks a lot. 🙏 I also tried http auth but didn't work.

bbot -t tesla.com -m httpx gowitness -c http_proxy=http://14ac7cb2f8d2c:4e90f0e479@91.190.191.94:12323

So, I guess there is no solution for this. I will run gowitness in separate scan without proxy.

[TheTechromancer]

I'm hoping this will get solved when we replace gowitness with playwright.

[TheTechromancer]

Closing this one. Please follow here https://github.com/blacklanternsecurity/bbot/discussions/698 for updates on the webscreenshot rewrite.