search

tomnomnom / meg

Fetch many paths for many hosts - without killing the hosts
MIT License
1.61k stars 268 forks source link

Support for storing response header only #32

Open ehsandeep opened 6 years ago

ehsandeep commented 6 years ago

Hi @tomnomnom,

Right now meg store response body for all the request, but sometimes we only look for response header for some header inspection, and we have no option to store the only header or exclude the body, will suggest to have optional flag for the same, as It can improve the speed in case we are only looking for response header.

leesoh commented 5 years ago

I think this feature will definitely save a bit of space, but I'm not so sure about saving time since the GET metho retrieves:

whatever information (in the form of an entity) is identified by the Request-URI.

Taken from RFC 2616. If you scroll down a page on that RFC though, you'll run into the HEAD request, which:

is identical to GET except that the server MUST NOT return a message-body in the response.

AND, a web server must support both GET and HEAD; all other methods are optional.

Does meg -X HEAD work for you?