search

postmodern / spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
MIT License
806 stars 109 forks source link

Link depth? #23

Closed ethicalhack3r closed 13 years ago

ethicalhack3r commented 13 years ago

Hi,

I am sure I saw somewhere that 'start_at' accepted a 'depth' parameter. I can't seem to find any reference to it any longer. Either way if I wasn't just imagining things, the depth option doesn't seem to be working. I grepped through the code and couldn't find it either.

Here is a pastie of a test spider of my blog: http://pastie.org/1754398

Is there a depth option? If so, am I using it wrong? or is it not working?

Thanks a million for any help. :)

postmodern commented 13 years ago

There is a :max_depth option in Git, thanks to nu7hatch. I suppose I should get on releasing Spidr 0.3.0.

ethicalhack3r commented 13 years ago

Ahhh... I was sure I saw it somewhere. Thanks for letting me know. :)

ethicalhack3r commented 13 years ago

Do you know if it would be possible for me to apply a patch to enable the max_depth option?

Thanks again, Ryan

postmodern commented 13 years ago

Impending release of Spidr 0.3.0.

postmodern commented 13 years ago

Spidr 0.3.0 released. Now with 100% more :max_depth.

ethicalhack3r commented 13 years ago

Awesome! Thank you! :D