search

jamesturk / spatula

A modern Python library for writing maintainable web scrapers.
https://jamesturk.github.io/spatula/
MIT License
244 stars 11 forks source link

multi-first-page scrapes #20

Open jamesturk opened 3 years ago

jamesturk commented 3 years ago

a canonical way to scrape two or more related pages as a combined entrypoint (HousePage & SenPage together for instance)

jefftriplett commented 3 years ago

This would be useful. I started with the test using --source option to test. I was a little surprised when I moved to scrape that it doesn't support --source. So having the ability to either use it or having multiple source options on the Page object would be useful.