search

internetarchive / brozzler

brozzler - distributed browser-based web crawler
Apache License 2.0
653 stars 96 forks source link

scroll down, and down, then scroll up #180

Closed galgeek closed 4 years ago

galgeek commented 4 years ago

Prioritize scrolling down in umbraBehavior, as does the custom facebook behavior.

I've checked the instagram behavior and @CorentinB's #168 with this update using brozzle_page. I've also started new test crawls of a facebook seed.