search

postmodern / spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
MIT License
800 stars 109 forks source link

Add default_headers option #47

Closed maccman closed 8 years ago

maccman commented 8 years ago

To allow people to pass in custom headers.

postmodern commented 8 years ago

Looks good!