Rate limiting and pagination

[ivan7707]

Hi,

I was taking a look through the code, but could not find anything pertaining to rate limiting and pagination. Does the code address either for the API?

Thanks.

[Brad-eki]

Hi Ivan,

From what I can tell the Harvest API (both v1 and v2) supports pagination.

When you list all of whatever object you are requesting there are parameters for page (which page you would like) and per_page (how many objects are listed per page).

I'm interested as to why rate limiting would be a concern.

Cheers

[Brad-eki]

Poking around for other reasons, I found this;

https://help.getharvest.com/api-v2/introduction/overview/pagination/

[Brad-eki]

also, just bumped into this:

https://help.getharvest.com/api-v2/introduction/overview/general/#rate-limiting

There is a limit on the Harvest end, around 100 requests per 15 seconds. I'm happy to be corrected but from what I can see the frequency with which you request cannot be policed by the python-harvest library as it is up to the calling program to trigger the HTTP request.

If you are hammering Harvest too much they will let you know; When the rate limit is exceeded Harvest will send an HTTP 429 status code.

[Brad-eki]

@ivan7707 , with respect to pagination - using invoices as an example;

The harvest method signature:

invoices(self, page=1, updated_since=None, status=None, from_date=None, to_date=None, client=None)

This means you can do something like: my_harvest = Harvest( connection parameters )

this_years_invoices = my_harvest.invoices(updated_since='2019-01-01')

But the expected response may contain more than 100 entries, so you might want to do this do this: this_years_invoices_page_1 = my_harvest.invoices(page=1, updated_since='2019-01-01') this_years_invoices_page_2 = my_harvest.invoices(page=2, updated_since='2019-01-01') Obviously that's for-loop territory.

If API v2 becomes supported, Harvest supports changing the number of entries on a page, but that will requires you to manage the pages.

In either case we are lucky - the response from the call to invoices has these elements in the data structure:

"per_page":100, "total_pages":1, "total_entries":2, "next_page":null, "previous_page":null, "page":1,

From that you can tell how many pages there are, which page you are on, and how many entries you have per page.

[Brad-eki]

If you are willing to use the Harvest API v2, I have committed some code which supports rate limiting. It is found in the v2_dev branch.