search

NikolaiT / GoogleScraper

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.
https://scrapeulous.com/
Apache License 2.0
2.64k stars 743 forks source link

UnicodeEncodeError: 'charmap' codec can't encode character #216

Open SnowHow21 opened 6 years ago

SnowHow21 commented 6 years ago

Hey, thanks for this awesome scraper, I appreciate it a lot. However, when I try to use keywords.txt or just a keyword such as "EXAMPLE, s.r.o., kontakt email tel. c." I get "UnicodeEncodeError: 'charmap' codec can't encode character '\u010d' in position 76: character maps to ". Further it shows "File "c:\users...\anaconda3\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0]" I have no idea what it could be but maybe itmight be some kind of problem with encoding? Thanks!

SnowHow21 commented 6 years ago

Okay, so there is no problem if the result is showed in the terminal or exported to anything else other than CSV. I also tried to change the encoding of the csv file to UTF-8 before running the scraper; however, it didn't help.

NikolaiT commented 6 years ago

Hey

Keyword files submitted with the command line argument --keyword-file KEYWORD_FILE must be UTF-8 encoded. Can you please try again and make sure your keyword file is in UTF-8?

Please pull the newest source from GoogleScraper.

Regards