Knowledge Sharing between packages

[dreyco676]

Hey @pat310 I maintain the following Python implementation of Google Trends API.

https://github.com/GeneralMills/pytrends

I really like what you've done with your JavaScript version. I just wanted to say that I will be keeping an eye on yours for new features to add to mine and that you should do the same!

In particular I've added suggestions() method that helps users refine their keyword searches by providing more detailed terms to use.

[pat310]

Thanks @dreyco676! That's a good idea, I will definitely check it out. I'll ping you whenever I add new features too.

[pat310]

@dreyco676 thanks for acknowledgement in the readme. I realized that I gave a shout out to your Python version in this quora a few months back: https://www.quora.com/Does-Google-Trends-have-a-publicly-available-API

Looking forward to checking it out in more detail.

[dreyco676]

@pat310 Google's API has been updated and the one we were using before now is severely rate limited. Check out the new endpoints in requests.py. Feel free to ping me with any questions.

[Dayjo]

@dreyco676 - Have you managed to overcome the rate limiting at all? I had an idea that I'm going to attempt in a week or two based on this; https://github.com/pat310/google-trends-api/issues/36 but no idea if it could be a reliable solution.

[dreyco676]

Yea so the rate limit is mostly due to the depreciated API. If you use the new one its in the hundreds per min. I honestly haven't hit it.

[Dayjo]

Are you also doing auth stuff though and sending the cookies? I noticed your python version makes you authenticate. Unfortunately this module doesn't send any cookies or do any auth at the moment so the rate limit is very tight.

[dreyco676]

Yea we are logging into Google and then using that as a session which keeps the cookies. Otherwise you hit the ratelimit pretty quick.

On Fri, Feb 3, 2017 at 11:14 AM, Joel notifications@github.com wrote:

Are you also doing auth stuff though and sending the cookies? I noticed your python version makes you authenticate. Unfortunately this module doesn't send any cookies or do any auth at the moment so the rate limit is very tight.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pat310/google-trends-api/issues/24#issuecomment-277305891, or mute the thread https://github.com/notifications/unsubscribe-auth/AGxCB16rKRFMEJldF33wxQQT54HS3cAWks5rY2BwgaJpZM4J0XbV .

[Dayjo]

That's the key I think. The plan will be to add an auth layer to this. Will attempt to delve into the python to see how you're doing it :) Thanks

[dreyco676]

http://stackoverflow.com/questions/6754709/logging-in-to-google-using-python

might be better documented than my script

[Dayjo]

Yeah, thanks (your script was perfectly understandable after I googled BeautifulSoup) I think the difficulty will be building it in a way that will work in generic Javascript and Node. Did you ever have any CORS issues?

[pat310]

@dreyco676 I think I got it working based on what you did in requests.py. Thanks!!

@Dayjo I'm probably going to do a lot of refactoring - will likely switch to es6 class structure and use a bundler like webpack. Thoughts?

[Dayjo]

@pat310 sounds like a plan! Never used webpack personally, but I'm sure it's easy enough :)

Let me know if you need a hand.

[pat310]

@Dayjo I've only checked one of the end points, but it looks like CORS is not enabled, so the library will have to remain a node application :disappointed:. I'll check some of the other endpoints to see if it would be possible to have any of the library work from the client

[Dayjo]

Ah damn, what's the difference between making a request from node, and making one from the browser?

Did you make any progress on the auth/cookies side of things?

[pat310]

@Dayjo Just published v4. Let me know if you run into any problems.

@dreyco676 Thanks for the heads up on the endpoint. I pointed out your library on the README.

[pat310]

closing this to clean up issues but definitely plan to continue knowledge sharing