search

hupili / python-for-data-and-media-communication-gitbook

An open source book on Python tailed for communication students with zero background
115 stars 62 forks source link

Pagination using HTTP header information (GitHub) #124

Open hupili opened 5 years ago

hupili commented 5 years ago

GitHub API has a special design. It returns the pagination information via HTTP header. Sometimes, one can not directly construct page skip URLs but one can continuously to "scroll back".

Example API: https://api.github.com/users/hupili/events/public?per_page=1

screenshot 2018-12-01 at 2 02 19 am

Those rel=next and rel=last URLs can be called.

hupili commented 5 years ago

Typical fail example:

https://api.github.com/users/hupili/events/public?page=400&per_page=1

screenshot 2018-12-01 at 2 05 07 am
ChicoXYC commented 5 years ago

try to understand and figure out the api, but still cannot get more than 300, may need your further instruction, Thanks --> https://github.com/hupili/github-activies/blob/master/github_activity.ipynb

hupili commented 5 years ago

what is the error when you reach 300?

p.s. above URL is invalid.

hupili commented 5 years ago

example: https://api.github.com/users/hupili/events/public?per_page=10&page=10

Link: <https://api.github.com/user/1227160/events/public?per_page=10&page=9>; rel="prev", <https://api.github.com/user/1227160/events/public?per_page=10&page=11>; rel="next", <https://api.github.com/user/1227160/events/public?per_page=10&page=30>; rel="last", <https://api.github.com/user/1227160/events/public?per_page=10&page=1>; rel="first"