issues
search
postmodern
/
spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
MIT License
800
stars
109
forks
source link
Add default_headers option
#47
Closed
maccman
closed
8 years ago
maccman
commented
8 years ago
To allow people to pass in custom headers.
postmodern
commented
8 years ago
Looks good!
To allow people to pass in custom headers.