search

rivermont / spidy

The simple, easy to use command line web crawler.
GNU General Public License v3.0
334 stars 69 forks source link

Use HTTP headers in HEAD Request #49

Closed AwesomeMarioFan closed 6 years ago

AwesomeMarioFan commented 6 years ago

This is a pull request to use the HTTP headers in both the HEAD and GET requests.