Can't Scrape News Type - Githubissues

xanderdunn commented 9 years ago

Running this command:

GoogleScraper -m http --keyword "apple" --search-type "news" --output-filename output.json -v2

produces this error:

Traceback (most recent call last):
  File "/usr/local/bin/GoogleScraper", line 9, in <module>
    load_entry_point('GoogleScraper==0.1.37', 'console_scripts', 'GoogleScraper')()
  File "/usr/local/lib/python3.4/site-packages/GoogleScraper/core.py", line 358, in main
    scrape_jobs = parse_all_cached_files(scrape_jobs, session, scraper_search)
  File "/usr/local/lib/python3.4/site-packages/GoogleScraper/caching.py", line 413, in parse_all_cached_files
    serp = parse_again(fname, job['search_engine'], job['scrape_method'], job['query'])
  File "/usr/local/lib/python3.4/site-packages/GoogleScraper/caching.py", line 443, in parse_again
    query=query
  File "/usr/local/lib/python3.4/site-packages/GoogleScraper/parsing.py", line 1003, in parse_serp
    parser = parser(query=query)
  File "/usr/local/lib/python3.4/site-packages/GoogleScraper/parsing.py", line 419, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.4/site-packages/GoogleScraper/parsing.py", line 89, in __init__
    self.__class__.__name__
AssertionError: search type "news" is not supported in GoogleParser

However, it should be supported, right? It's listed in the help message for the supported types:

GoogleScraper - WARNING - Invalid search type! Select one of ('normal', 'video', 'news', 'image')

It looks like it's not listed as a supported search type for the Google Parser

Is news scraping not yet supported, despite being listed?

steve21124 commented 9 years ago

agree. not able to scrap google news yet.

Vineeth-Mohan commented 8 years ago

Kindly fix this

JimmySpivey commented 7 years ago

I am interested in implementing this.

metromark commented 5 years ago

Any updates on this so far?

NikolaiT / GoogleScraper

Can't Scrape News Type #110