I'm currently working on a scraping a website (sorry, can't share it at the moment) that handles pagination more like offset in SQL than using page numbers. Each page displays 20 instances. The first page is start=1, the second page is start=21, the third is start=41, etc. This pull request allows the user to ability to specify a number other than 1 that should be added to pagination_index to get the next page. The default value of pagination_interval is 1. For this site, setting up Upton to handle pagination would like this:
I'm currently working on a scraping a website (sorry, can't share it at the moment) that handles pagination more like
offset
in SQL than using page numbers. Each page displays 20 instances. The first page isstart=1
, the second page isstart=21
, the third isstart=41
, etc. This pull request allows the user to ability to specify a number other than 1 that should be added topagination_index
to get the next page. The default value ofpagination_interval
is 1. For this site, setting up Upton to handle pagination would like this: