A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
MIT License
805
stars
109
forks
source link
Check for opaque part of URI before attempting to set the path #65
Fixes https://github.com/postmodern/spidr/issues/57