A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Add optional Logging/debug output to Spidr::Agent. Agent#initialize should accept a logger option for passing in custom Logger compatible objects. It should also support a logging: true|false option, which initializes @logger to a new Logger instance.
Add optional Logging/debug output to
Spidr::Agent.Agent#initializeshould accept aloggeroption for passing in custom Logger compatible objects. It should also support alogging: true|falseoption, which initializes@loggerto a newLoggerinstance.