search

postmodern / spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
MIT License
800 stars 109 forks source link

Switch to `Addressable::URI` for URI parsing #86

Open postmodern opened 1 year ago

postmodern commented 1 year ago

Using Addressable::URI would allow spidr to handle IDN domains.