Single URL errors - Githubissues

sensepost / gowitness

🔍 gowitness - a golang, web screenshot utility using Chrome Headless

GNU General Public License v3.0

2.82k stars 323 forks source link

Single URL errors #66

Closed znre closed 2 years ago

znre commented 3 years ago

Describe the bug I am experiencing two errors when trying to run gowitness. One error says something about a "context deadline exceeded", and the other is "failed to save screenshot buffer"

To Reproduce Steps to reproduce the behavior: On the command line, type:

gowitness single https://tesla.com
gowitness single https://reddit.com

Expected behavior Screenshots will be taken of the site.

Screenshots

Version Information:

OS: Kali GNU/Linux Rolling x86_64
gowitness: 2.0.0

randomactsofsecurity commented 3 years ago

Ran into this issue as well recently, think the issue may be from the instantiation of the http transport in chrome.go missing some of the defaults?

transport := http.DefaultTransport.(*http.Transport).Clone()

Seems to fix the issue with a shallow copy, but need to test it out a bit more against more assets in comparison to the old version.

leonjza commented 3 years ago

I can replicate this on Kali for https://tesla.com, but works fine on macOS. 🤔

leonjza commented 3 years ago

So there are two problems here. For the error related to reddit a fix is incoming shortly. As for tesla.com, there seems to be weirdness in how the server responds based on the User-Agent and Connection headers. Still debugging this.

leonjza commented 3 years ago

I am happy I fixed the reddit example (thanks for reporting @znre). As for tesla.com, the preflight request fails with this request.

For the sake of copy/paste.

GET / HTTP/1.1
Host: www.tesla.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36
Referer: https://tesla.com
Accept-Encoding: gzip, deflate
Connection: close

Change the User-Agent to curl, and it will work, or Connection to keep-alive and it will sometimes work. So the question is I guess, does it make sense to set the UA in the preflight as well? In my mind it does, but we have this interesting case now.

artemidas commented 3 years ago

Hey there, any updates on this one? It seems that gowitness single https://www.tesla.com/es_ES/ works just fine. I'm from Spain by the way 😉

lyonys commented 3 years ago

@leonjza I am still facing the same issues in the latest build. gowitness fails to create a local 'screenshot' folder. I think we can change it to use mkdirall instead of mkdir and it would resolve the issue.

https://stackoverflow.com/questions/37932551/mkdir-if-not-exists-using-golang

leonjza commented 3 years ago

@lyonys are you providing a path that is deeper than 1 directory? Right now we are preparing the directory here.

leonjza commented 2 years ago

Closing this for now. Looks like the tesla.com example works at the moment.

Samwise-G commented 9 months ago

only the https://tesla.com is not working again, reproduced on kali