Open josh-ashkinaze opened 1 month ago
the same
Same for me, It appers that only a few methods keep working.
The following are working
Suggestions & Categories are not performing that well
build_payload() doesn't seem to be working for me
How you're doing it?
it's same for me I've been attempting to make API calls using PyTrends, but I keep receiving an HTTP error 429, which indicates "Too Many Requests." This error typically occurs when the rate limit for API requests is exceeded. However, the strange part is that I encounter this error even when I make only a single API call.
the same here =/
Same for me, It appers that only a few methods keep working.
The following are working
- realtime_trending_searches()
- trending_searches(pn='united_states')
- suggestions(keyword='coronavirus')
- categories()
Suggestions & Categories are not performing that well
Can I ask what payload you are using please?
Same for me, It appers that only a few methods keep working. The following are working
- realtime_trending_searches()
- trending_searches(pn='united_states')
- suggestions(keyword='coronavirus')
- categories()
Suggestions & Categories are not performing that well
Can I ask what payload you are using please?
Sure!, here it is
pytrend = TrendReq(hl='en-US', tz=360, retries=3)
pytrend.build_payload(kw_list=['whatever word you want'])
-To get realtime trendings
df = pytrend.realtime_trending_searches(pn='US')
df.head(10)
-To get top searches in US
df = pytrend.trending_searches(pn='united_states')
df.head(10)
-To get suggestions based on a word
df = pytrend.suggestions(keyword='obamacare')
print(df)
-To get related categories
df = pytrend.categories()
print(df)
df = pytrend.today_searches(pn='US')
df.head(2)
Please note that these methods are working but some others are not. This is because Pytrends is not an official API, it is an open source project that actually scraps the data from Google Trends, this implies that often we may find errors in Pytrends every time Google change something in the Google Trend's Page
Same for me, It appers that only a few methods keep working. The following are working
- realtime_trending_searches()
- trending_searches(pn='united_states')
- suggestions(keyword='coronavirus')
- categories()
Suggestions & Categories are not performing that well
Can I ask what payload you are using please?
Sure!, here it is
pytrend = TrendReq(hl='en-US', tz=360, retries=3) pytrend.build_payload(kw_list=['whatever word you want'])
-To get realtime trendings
df = pytrend.realtime_trending_searches(pn='US') df.head(10)
-To get top searches in US
df = pytrend.trending_searches(pn='united_states') df.head(10)
-To get suggestions based on a word
df = pytrend.suggestions(keyword='obamacare') print(df)
-To get related categories
df = pytrend.categories() print(df)
- To get today's searches in US
df = pytrend.today_searches(pn='US') df.head(2)
Please note that these methods are working but some others are not. This is because Pytrends is not an official API, it is an open source project that actually scraps the data from Google Trends, this implies that often we may find errors in Pytrends every time Google change something in the Google Trend's Page
@lapiceroazul4 - Thank you very much.
Are the other methods being fixed currently, or does anyone know of good ways to work around them? I'm trying to use interest_over_time() and it seems to have completely stopped working, probably when Google Trends updated recently. I've been consistently getting 429 errors on my first request, and haven't had luck with changing backoff_factor/retries, using proxies, or even running it on a different computer and network.
Starting to see data coming in now, but slowly. No immediate 429.
I am also having the "TooManyRequestsError: The request failed: Google returned a response with code 429" error
How the hell are these guys able to do it? 🤔 https://serpapi.com/playground?engine=google_trends&q=Coffee
interest_over_time() only returns 429 for me also
How the hell are these guys able to do it? 🤔 https://serpapi.com/playground?engine=google_trends&q=Coffee
Idk how they do that, but, it looks awesome, any idea of how can we get that data?
How the hell are these guys able to do it? 🤔 https://serpapi.com/playground?engine=google_trends&q=Coffee
Idk how they do that, but, it looks awesome, any idea of how can we get that data?
hmm..
How the hell are these guys able to do it? 🤔 https://serpapi.com/playground?engine=google_trends&q=Coffee
Idk how they do that, but, it looks awesome, any idea of how can we get that data?
hmm..
Got any idea?
I just know, that if I try to replicate the http calls my browser does in postman, I keep getting an error that this isn't a browser. So my guess would be, that they are just web scraping it. When you use their API (with only 100 free calls/months) my guess would be, that they have headless browsers running on a server.
Can this library include such a feature where it uses a headless browser to do the scraping just like the API? Possibly, someone can add it
On Sat, Aug 17, 2024 at 2:25 PM Tobias @.***> wrote:
I just know, that if I try to replicate the http calls my browser does in postman, I keep getting an error that this isn't a browser. So my guess would be, that they are just web scraping it. When you use their API (with only 100 free calls/months) my guess would be, that they have headless browsers running on a server.
— Reply to this email directly, view it on GitHub https://github.com/GeneralMills/pytrends/issues/625#issuecomment-2294935238, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7IJS7IH2BKR6FLQKDN3BTZR6ISBAVCNFSM6AAAAABK3TZXKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJUHEZTKMRTHA . You are receiving this because you are subscribed to this thread.Message ID: @.***>
My personal workaround for similar data is to use the wikipedia api: see python wikipedia api for page views
Here is a MRE, using what is in README
This throws a 429 errors. I tried wrapping this with tenacity and I keep getting 429s no matter how long I wait.