search

casmlab / get-nytimes-articles

Python tools for getting data from the New York Times Article API. Retrieves JSON from the API, stores it, parses it into a CSV file.
MIT License
46 stars 25 forks source link

Handle 403 errors gracefully #4

Closed libbyh closed 7 years ago

libbyh commented 7 years ago

HTTP 403 Forbidden errors occur, probably because of API call limits. Script should fail gracefully - tell user what's happening and why. Currently does so in logs, but people aren't always looking there.

libbyh commented 7 years ago

Doesn't seem to be a rate limit issue after all. I can submit another request immediately and get results. Still need to handle the 403.

libbyh commented 7 years ago

Example of API calls that produce 403 error:

http://api.nytimes.com/svc/search/v2/articlesearch.json?begin_date=19900102&end_date=19900102&page=17&api-key=[removed]

http://api.nytimes.com/svc/search/v2/articlesearch.json?begin_date=19900106&end_date=19900106&page=16&api-key=[removed]

In both cases, the pages before return results, and the page after returns OK but no more articles. Is this a math problem?