search

openplans / openblock

OpenBlock is a web application and RESTful service that allows users to browse and search their local area for "hyper-local news
61 stars 26 forks source link

georeport v2 scraper needs to do pagination #238

Closed slinkp closed 11 years ago

slinkp commented 11 years ago

Pagination is not part of the spec, but per discussion on the open311 mailing list, several georeport v2 servers have implemented a 'page' parameter, and seeclickfix has an implicit pagesize of 20 (violating the spec which says 1000 items by default). Looks like 'page' is going to end up as a backward-compatible add-on parameter in the 2.1 spec.

So to get data from those we need to handle pagination. Just pass a page parameter, and if we get nothing back from a page, stop.

slinkp commented 11 years ago

(In [6acd23811bed99a66f959f751f9fa137465052a4]) Support pagination in georeport v2 scraper; closes #245

slinkp commented 11 years ago

Ticket imported from Trac: http://developer.openblockproject.org/ticket/245 Reported by: slinkp