[bug] Related queries does not work

[josh-ashkinaze]

Thanks for working on this now that it seems pytrends is down!

Here is a MRE that I was running on Google Colab

!pip install trendspy
from trendspy import Trends
tr = Trends()
related = tr.related_queries('python')

related returns nothing:

# print(related)

{'top': Empty DataFrame
 Columns: []
 Index: [],
 'rising': Empty DataFrame
 Columns: []
 Index: []}

EDIT: Latest version on GitHub does not work either, so no commits in between public release fixed this.

!pip install --upgrade git+https://github.com/sdil87/trendspy.git

from trendspy import Trends
tr = Trends()
related = tr.related_queries(['python'])
print(related)

[sdil87]

Unfortunately, the related_queries and related_topics functions are very sensitive to Google's quota limits. Here's an example to illustrate the issue:

from trendspy import Trends
from trendspy.converter import TrendsDataConverter

# Using a localized token
tr = Trends(hl='en-US')
token, data = tr.related_queries('python', return_raw=True)

# Check user type in the token
print(token['request']['userConfig']['userType']) 
# Output can be: USER_TYPE_EMBED (working) or USER_TYPE_EMBED_OVER_QUOTA

# Convert related queries to DataFrame
df = TrendsDataConverter.related_queries(data)
print(df)

# Example output:
# {'top': 
#                      query  value
#  0               python for    100
#  1              python list     74
#  2              python code     60
#  3            python string     56
# }

Using the default token (hl='en-US'), I received USER_TYPE_EMBED_OVER_QUOTA. Changing the hl parameter to match my locale resolved the issue temporarily. Separately, after interacting with Google Trends in my browser, the queries started working again.

This is not a bug in the library but rather a limitation imposed by Google. I would also love to understand these limits better and determine the conditions under which it starts working again. Any help in tracing these patterns would be greatly appreciated!

[sdil87]

Hi @josh-ashkinaze! I found a way to work around this limitation in version 0.1.4, and it should now work much better. I tested it with 600 sequential requests (with 1-second pauses), and it performed smoothly. Just a heads-up: if you send too many requests without pauses, you might run into a 429 error. Let me know how it works for you now! 😊