search

xmunoz / sodapy

Python client for the Socrata Open Data API
MIT License
402 stars 114 forks source link

support automatic pagination #69

Closed afeld closed 4 years ago

afeld commented 4 years ago

Hey, nice library! I was thinking, it would be useful for the package to support pagination for get() (maybe other methods), to make it easier to retrieve large datasets. Maybe an optional paginate=True keyword argument? Thanks!

xmunoz commented 4 years ago

As it stands now, you can implement your own pagination with the $limit and $offset SoQL keywords. If you want to take a stab at implementing something like this with a default limit (1K records?), I think that would be be a real benefit for many users of this library.