search

pat310 / google-trends-api

An API layer on top of google trends
https://www.npmjs.com/package/google-trends-api
MIT License
889 stars 176 forks source link

Add trending searches method #99

Open tobihagemann opened 6 years ago

tobihagemann commented 6 years ago

Have you already thought about adding a "trending searches" method, like in the Python library you've linked? https://github.com/GeneralMills/pytrends#trending-searches

pat310 commented 6 years ago

That could be added! Looks like it would take a bit of configuration changing since it requires POST vs GET, but definitely doable

Colibry-Source commented 5 years ago

Hi there ...

I need the trending search methods and would love to use your framework to implement these apis for the missing search(es). Do you mind if I add/participate on your project, this would be the ideal place for the method(s)?

Please let me know. In the meanwhile I'll research what is necessary.

Regards,

Patrick JP Colibry Consulting

pat310 commented 5 years ago

@Colibry-Source That would be great. Would you be able to make PRs in?

Colibry-Source commented 5 years ago

Hi, sorry I don't understand what you mean by 'PRs'? Do you mean a pull Request?

pat310 commented 5 years ago

Yup, pull requests

pat310 commented 5 years ago

@Colibry-Source It looks like the python trending searches uses this URL with a POST method: TRENDING_SEARCHES_URL = 'https://trends.google.com/trends/hottrends/hotItems'

Right now in utilities.js it's hard coded to always perform a GET so we would need to update this object to also take a method

pat310 commented 5 years ago

@Colibry-Source I think we should look into the daily trends endpoint, it would probably be easier to implement and might cover what we're looking for.

It goes something like this: https://trends.google.com/trends/api/dailytrends?hl=en-US&tz=300&geo=US&ns=15

Let me know what you think.

pat310 commented 5 years ago

There is also real time searches: https://trends.google.com/trends/api/realtimetrends?hl=en-US&tz=300&cat=all&fi=0&fs=0&geo=US&ri=300&rs=20&sort=0

Colibry-Source commented 5 years ago

Thanks for bringing this to my attention. I'll add it to the api also.

PJP

⁣Sent from BlueMail ​

On Dec 2, 2018, 13:34, at 13:34, Patrick Trasborg notifications@github.com wrote:

There is also real time searches: https://trends.google.com/trends/api/realtimetrends?hl=en-US&tz=300&cat=all&fi=0&fs=0&geo=US&ri=300&rs=20&sort=0

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/pat310/google-trends-api/issues/99#issuecomment-443531056

Colibry-Source commented 5 years ago

Yes, this is exactly what I'm using to implement. It fits nicely with the framework you've created and as such I will avoid the extra work of implementing post logic. I did not find a reference to the POST method you talked about. I wanted to do some research to see if there was a difference in results. I'll be moving ahead with the GET solution in the meanwhile.

On Sun, Dec 2, 2018 at 1:29 PM Patrick Trasborg notifications@github.com wrote:

@Colibry-Source https://github.com/Colibry-Source I think we should look into the daily trends https://trends.google.com/trends/trendingsearches/daily?geo=US endpoint, it would probably be easier to implement and might cover what we're looking for.

It goes something like this:

https://trends.google.com/trends/api/dailytrends?hl=en-US&tz=300&geo=US&ns=15

Let me know what you think.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pat310/google-trends-api/issues/99#issuecomment-443530659, or mute the thread https://github.com/notifications/unsubscribe-auth/ApknalTaNj9lzxMyow2pICmdXsNPGgwBks5u1Bv1gaJpZM4Syjg7 .

Colibry-Source commented 5 years ago

Hi Patrick,

Just a quick update. I was able to implement both in a way that seems to compliment the existing work. However, the returned data set for both queries are very verbose. I'll provide some more info in the documentation on how to use it. The real-time query search in particular will give around 8 results in detail, but will provide a list of 300 articles IDS that represent trending information. I have no found a way to use theses articles IDS to pull back more information if the 8 provided are not enough. Unfortunately, I don't have the time to research this further at the moment as I am behind in my own project that uses this information. I see somehow the official googletrends page is translating these to another ID and them doing a 'post' call to retrieve more information with a whole bunch of information sent that I have no idea how is gotten but I can confirm that the info passed isn't present in the result set that was provided by the initial google trend call. So there is a lookup "process" that is happening that I haven't been able to identify. Perhaps Ill look at the python version of these libraries (that seem to be broken also) to see what was done in the past.

The next steps, once i have some down time would be to update the tests, document and research more behavior about the needed parameters and then you can approve and merge it back into the project.

I'll update my own repository with the preliminary code and you can give it a look over.

Regards,

PJP

On Mon, Dec 3, 2018 at 4:12 PM Colibry Consulting < colibryconsulting@gmail.com> wrote:

Yes, this is exactly what I'm using to implement. It fits nicely with the framework you've created and as such I will avoid the extra work of implementing post logic. I did not find a reference to the POST method you talked about. I wanted to do some research to see if there was a difference in results. I'll be moving ahead with the GET solution in the meanwhile.

On Sun, Dec 2, 2018 at 1:29 PM Patrick Trasborg notifications@github.com wrote:

@Colibry-Source https://github.com/Colibry-Source I think we should look into the daily trends https://trends.google.com/trends/trendingsearches/daily?geo=US endpoint, it would probably be easier to implement and might cover what we're looking for.

It goes something like this:

https://trends.google.com/trends/api/dailytrends?hl=en-US&tz=300&geo=US&ns=15

Let me know what you think.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pat310/google-trends-api/issues/99#issuecomment-443530659, or mute the thread https://github.com/notifications/unsubscribe-auth/ApknalTaNj9lzxMyow2pICmdXsNPGgwBks5u1Bv1gaJpZM4Syjg7 .

Colibry-Source commented 5 years ago

The proposed source changes can be found here:

https://github.com/Colibry-Source/new-google-trends-api

You'd need to run the build process script via npm , and then use the outputted library to test the newer apis.

No documentation for it yet but here is my entry code that you can run with:

googleTrends.realTimeTrends({ geo: 'US', category: 'all', }, function(err, results) { if (err) console.log('oh no error!', err); else console.log(results); });

googleTrends.dailyTrends({ trendDate: new Date('2018-12-15'), geo: 'US', }, function(err, results) { if (err) { console.log('oh no error!', err) }else{ console.log(results); } });

Tests updates to come later but feel free to add feedback.

Colibry-Source commented 5 years ago

Sorry i thought it best to close the old pull request because I had to remerge with with your master and then add some additional code based on the changes that were added in version 4.8. I hope this makes it cleaner to look at the newer code overall.

HyperionZhou commented 2 years ago

There is also real time searches: https://trends.google.com/trends/api/realtimetrends?hl=en-US&tz=300&cat=all&fi=0&fs=0&geo=US&ri=300&rs=20&sort=0

Thank you very much! I am just looking for this link!