sensepost / gowitness

🔍 gowitness - a golang, web screenshot utility using Chrome Headless
GNU General Public License v3.0
2.87k stars 329 forks source link

fix: Allow wappalyzer to correctly use the response body #127

Closed terrabitz closed 2 years ago

terrabitz commented 2 years ago

The Issue

In the master branch of gowitness, the wappalyzer module is used to detect technologies used by a scanned website. However, I found that not all the technologies on the website were being discovered correctly.

After some investigation, it appeared that the wappalyzer.Fingerprint command was always getting a zero-length byte array as the body argument. This was due to the http.Response.Body already being read previously during GetHTMLTitle. When a http.Response.Body is read once, it's no longer usable and should be closed; any further reads result in an empty byte array.

The Fix

Instead of passing the response body around and allowing subfunctions to read it, it's read once as part of chrome.Preflight. That byte array is then passed to the relevant subfunctions.

Test Methodology

Perform a scan using the code from the latest master branch, and another using the modified version. Validate that more technologies are identified in the latter.

Screenshots

Before the fix: 2022-02-17-15-51-45

After the fix: 2022-02-17-15-52-55

leonjza commented 2 years ago

Thanks for the PR! I was behind on the PR stack for this project, and having since caught up a bit this PR now has merge conflicts. Apologies for that! Would you mind rebasing?

terrabitz commented 2 years ago

@leonjza I just rebased, accounting for the PR that switches to a singleton Wappalyzer client.

leonjza commented 2 years ago

Thanks!