The request failed: Google returned a response with code 500.

[cent664]

Hi,

I"m trying to scrape 365 days of data points from today (2019-12-09 to 2020-03-18). I can't seem to scrape data from somewhere around the dates 2020-02-27 to 2020-03-05. I've tried different keywords, same thing.

I tried splitting the query into two parts: One from 2019-12-09 to 2020-02-27 and Second from 2020-02-27 to 2020-03-18

That successfully made the query run (no code 500), but returned an empty dataframe for the problem time range (2020-02-27 to 2020-03-05).

The worst part is, when I scraped last friday (2020-03-13) for 365 days (which included the problematic data range), it worked fine - no empty dataframe. I have the latest pandas and pytrends.

PS: I can attach the whole python file if you guys want, just felt like it might be unnecessary.

Please help. I have no idea how and why this is happening. My code is as follows:

`def get_trends(keyword, days): i = 0 waiting_time = 60

"""Specify start and end date as well es the required keyword for your query"""
print("Trend keyword: ", type(keyword))

"""Calculating start date from end date, based on given interval"""
end_date = datetime.datetime.date(datetime.datetime.now())
# end_date = end_date - datetime.timedelta(days=20)
print("Today's date: ", end_date)
start_date = end_date - datetime.timedelta(days=days)
print(start_date, end_date)

"""Since we want weekly data for our query, we will create lists which include
the weekly start and end date."""

weekly_date_list = []

# Adds the start date as first entry in our weekly_date_list
start_date_temp = start_date
weekly_date_list.append(start_date_temp)

# This will return in list of weekly datetime.date objects - except the end date
while start_date_temp + datetime.timedelta(days=7) <= end_date:
    start_date_temp += datetime.timedelta(days=7)
    weekly_date_list.append(start_date_temp)

# This will add the end date to the weekly_date list. We now have a complete list in the specified time frame
if start_date_temp + datetime.timedelta(days=7) > end_date:
    weekly_date_list.append(end_date)

print(weekly_date_list)

"""Now we can start to downloading the data via Google Trends API
therefore we have to specify a key which includes the start date
and the end-date with T00 as string for hourly data request"""

"""This list will contain pandas Dataframes of weekly data with the features "date",
"keyword"(which contains weekly scaling between 0 and 100), "isPartial".
Up to this point, the scaling is not correct."""

interest_list = []

# Here we download the data and print the current status of the process
while i < len(weekly_date_list) - 1:
    key = str(weekly_date_list[i]) + "T00 " + str(weekly_date_list[i+1]) + "T00"
    p = TrendReq()
    p.build_payload(kw_list=[keyword], timeframe=key)
    interest = p.interest_over_time()
    interest_list.append(interest)
    print("GoogleTrends Call {} of {} : Timeframe: {} ".format(i + 1, len(weekly_date_list) - 1, key))
    i += 1

# print(interest_list)`

[TarmacD]

I also have to struggle with the problem that in March I only get response 500 for some weeks, but in all other weeks before and after (also last week) everything works fine. I cannot find out why...

[cent664]

I also have to struggle with the problem that in March I only get response 500 for some weeks, but in all other weeks before and after (also last week) everything works fine. I cannot find out why...

That period of time in early march seems to be the problem.

[praveenmathew93]

Hello guys, any idea why this happens. I was working for me till yesterday and suddenlt, BAM!. The same code has been giving this error. I thought maybe Google thought I am a suspicious client. I used VPN and tried, still the same. Tried from a different system connected to a different netword, still the error keeps coming.