search

sensepost / gowitness

🔍 gowitness - a golang, web screenshot utility using Chrome Headless
GNU General Public License v3.0
2.87k stars 329 forks source link

Gowitness runs between 150-300 results then either stops or just says 'killed' #81

Closed joseph-giron closed 3 years ago

joseph-giron commented 3 years ago

So I'm trying to produce a bunch of screenshot and I have something like 45k hosts to go through, but gowitness will only process the first 150-300 hosts before dying. I've enabled debugging, but that doesn't show it.

I'm on ubuntu 20 on Digital Ocean. I'm running as root so I thought maybe it might be a permissions issue, so I added a non root user and ran gowiness from that, but no dice.

Is there a way I can set chrome flags directly from gowitness? I didn't see an option within chromedp. maybe I could disable the sandbox or something.

Version Information:

Additional context Works fine on my laptop and desktop running chrome / gowitness. Not sure why it won't work on a VPS / CLI only environment.

leonjza commented 3 years ago

Could it be the OOM killing gowitness here? How much memory does the droplet have and how many goroutines are you using (-t flag)? Maybe try and reduce those a bit on the droplet?

I rate a more interesting answer would be to know how much the memory footprint grows over time; maybe the garbage collector is having a hard time.

joseph-giron commented 3 years ago

You know...that might be it. It would make the most sense. Thanks for the quick response.