Grepable output on the screenshot feature

AlessandroZ commented 2 months ago

Hello,

I like the "screenshot" feature, not only to have screenshots but also to have responses in plain text. However, it would be nice to have a grepable output on the txt file to be able to parse results.

For example, right now we have:

GET / HTTP/1.1
Host: xxxxxxxx
User-Agent: xxxxxxx
Accept-Charset: utf-8
Accept-Encoding: gzip

HTTP/1.1 200 OK
Connection: close
Cache-Control: private, max-age=0, no-cache, no-store
Content-Type: text/html
Pragma: no-cache

<html>
....
</html>

To something like meg (https://github.com/tomnomnom/meg)

> GET / HTTP/1.1
> Host: xxxxxxxx
> User-Agent: xxxxxxx
> Accept-Charset: utf-8
> Accept-Encoding: gzip

< HTTP/1.1 200 OK
< Connection: close
< Cache-Control: private, max-age=0, no-cache, no-store
< Content-Type: text/html
< Pragma: no-cache

<html>
....
</html>

I will also separate the screenshot functionality to the output text file, depending on the needs, sometimes the screenshot is not necessary.

Thank you again for your great work !!

Have a nice day.

GeorginaReeder commented 2 months ago

Thanks for your suggestion @AlessandroZ , we'll take a look into this!

dogancanbakir commented 2 months ago

@AlessandroZ, Can you provide some use cases to help us understand why this would be better - and the current output isn't? Thanks!

AlessandroZ commented 2 months ago

I like to grep for header response or at least listing it to detect custom headers used by applications. Right now, I cannot determine where are headers from the response. With another output, I could grep for "< " for example, and then do my checks.

As you write the output to a file, it could be easy to format it. Or separate, the request to the response, or separate the response body to the response header. Many ways are possible.

If for you there is not interest to implement, I could do using other ways (using meg for example).

dogancanbakir commented 2 months ago

I'm interested, and that's why I'm asking these questions to better understand the problem. There's a flag that can provide you with response headers. Please let me know if that will solve the problem. If not, we can discuss whether adding this is necessary. The flag:

$ httpx -u scanme.sh -silent -j -irh | jq .header
{
  "content_length": "2",
  "content_type": "text/plain; charset=utf-8",
  "date": "Mon, 22 Jul 2024 08:43:49 GMT"
}

AlessandroZ commented 2 months ago

Hi, yes I didn't know this feature. Sorry to miss it. It could do the job.

Thank you for your help.

projectdiscovery / httpx

Grepable output on the screenshot feature #1831