Download of schedule/stop pairs fails if amount exceeds PER_PAGE

[BurnsPoint]

Hi there! Just discovered this project — so awesome, thanks! However I'm finding that it fails on larger datasets. The issue appears to be with how the transitland_api.py handles pagination.

For example, the metro rail of Wellington, New Zealand: If I run

transitflow sam$ python3 transitflow.py --name=wellington_rail --operator=o-rbs-metlinkrail --date=2019-09-09

I can see that there are 10 routes, 62 stops and 3806 schedule stop pairs, and I get no issue.

Now I go to line 17 in transitflow.py and change PER_PAGE from 10000 to 3805, rerun the above command and get a download error:

transitflow sam$ python3 transitflow.py --name=wellington_rail --operator=o-rbs-metlinkrail --date=2019-09-09
INPUTS:
date:  2019-09-09
name:  wellington_rail
API key:  None
1 operators to be downloaded.
b'o-rbs-metlinkrail' 1 / 1
http://transit.land/api/v1/routes?operated_by=o-rbs-metlinkrail&per_page=3805
10 routes found.

http://transit.land/api/v1/stops?served_by=o-rbs-metlinkrail&per_page=3805
62 stops found.

http://transit.land/api/v1/schedule_stop_pairs?operator_onestop_id=o-rbs-metlinkrail&date=2019-09-09&per_page=3805&sort_min_id=0
https://api.transit.land/api/v1/schedule_stop_pairs?date=2019-09-09&operator_onestop_id=o-rbs-metlinkrail&per_page=3805&sort_key=id&sort_min_id=4923869135&sort_order=asc
failed:
0 operators successfully downloaded.
1 operators failed.
failed operators: [b'o-rbs-metlinkrail']

But if I change PER_PAGE to 3806 then there's no issue either. So it appears when the amount of schedule/stop pairs exceeds per_page limit then the download breaks... :(

Also, both those URLS for getting the schedule/stop data are fine. Will have a crack at fixing myself but just putting this out there is case anyone has any ideas!

[BurnsPoint]

Oh don't worry – was just because the next_uri given by transitland is now an https address and I didn't have my certs configured right! 🙃