search

alirezamika / autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python
MIT License
6.24k stars 654 forks source link

Google Trend Scraping #61

Closed rushiai closed 2 years ago

rushiai commented 3 years ago 
url="https://trends.google.com/trends/trendingsearches/realtime?geo=IN&category=all"
wanted_list= ['Tamil Nadu • Lockdown • Coronavirus','https://www.sentinelassam.com/national-news/tn-police-makes-ockdown-violators-sit-with-fake-covid-positive-patients-on-their-two-wheelers-539462']
scraper=AutoScraper()
result=scraper.build(url,wanted_list)
print(result)

Output >> [ ]

alirezamika commented 2 years ago

this page uses js to fetch data, you can use tools like selenium to get the page data and pass it to autoscraper.