ruipgil / scraperjs

A complete and versatile web scraper.
MIT License
3.71k stars 188 forks source link

Character encoding - possible to set it? #65

Open mesic opened 8 years ago

mesic commented 8 years ago

I'm trying to parse a website however the characters are read wrong, probably due to the character setting. Is there any way this can be set?

ruipgil commented 7 years ago

Yes, scraperjs uses request, you should refer back to it.

PauloAmaroRamos commented 6 years ago

Instead of a string for the url: scraperjs.StaticScraper.create('http://www.somesite.com').scrape(...)

Use an object with the parameters url and encoding, like this: scraperjs.StaticScraper.create({ url: 'http://www.somesite.com', encoding: "binary" }).scrape(...)