search

postmodern / spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
MIT License
800 stars 109 forks source link

Add ignore_paths and ignore_paths_like #61

Open postmodern opened 7 years ago

postmodern commented 7 years ago

Add methods/options for filtering URLs by path.