search

postmodern / spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
MIT License
800 stars 109 forks source link

Session handling #53

Closed heavysixer closed 7 years ago

heavysixer commented 7 years ago

Is it possible for Spidr to follow some setup param like visiting a page and logging in before spidering the remaining page?

postmodern commented 7 years ago

@heavysixer yes, spidr already supports a basic Cookie Jar. https://github.com/postmodern/spidr/blob/master/lib/spidr/cookie_jar.rb