GeneralMills / pytrends

Pseudo API for Google Trends
Other
3.21k stars 813 forks source link

various inconsistencies in behavior and data retrieval for get_historical_interest #371

Open scott2b opened 4 years ago

scott2b commented 4 years ago

get_historical_interest sometimes throws a 500 error, other times returns data entries of 0 where data is known to exist and can be retrieved via other requests with overlapping dates. At other times it simply returns no data.

Behavior is too inconsistent to definitely report a reproducible pattern here. But here are some examples I am seeing. Using this basic code construct:

keywords = ['Chipotle']
pt = TrendReq(tz=0, timeout=(10,25), retries=2, backoff_factor=0.1)
start_day = 9
end_day = 11
pt.get_historical_interest(keywords,
                           year_start=2020,
                           month_start=3,
                           day_start=start_day,
                           hour_start=0,
                           year_end=2020,
                           month_end=3,
                           day_end=end_day,
                           hour_end=23,
                           sleep=30)

With various changes to start_day and end_day, I get the following:

start_day, end_day: result 4, 7: 500 error (The request failed: Google returned a response with code 500) and empty frame 4, 11: 500 error, returns a frame but only data for the 11th 9, 11: zeroes for all the values 10, 11: valid response (including data for the 11th that was previously all zeroes) 5, 11: empty frame (not zeroes, just empty)

I have yet to discover any particular pattern in the behaviors. I only note that there are various combinations of 500, populated and empty frames, and zero data vs valid data. The results are consistent for a given date range if the same request is repeated, but for various date ranges, the results could be any combination of the aforementioned, including inconsistent results for a given date.

manisci commented 2 years ago

I am having the same problem too :(