search

shaikhsajid1111 / facebook_page_scraper

Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
https://pypi.org/project/facebook-page-scraper/
MIT License
222 stars 67 forks source link

Language support for other languages (not english) #97

Open Janithpm opened 7 months ago

Janithpm commented 7 months ago

Some of posts I'm scraping has sentences in another language (specifically in the Sinhala language). when export as a csv it's not showing as symbols. is there any feature to support all languages?

Janithpm commented 7 months ago

this is one line of CSV file

pfbid02zX2FVAaCwr8TWQmE9u4p8eW2jTfc7dWteG4m1R3zygpnrFn4smpJ6EyFgKRvi7UTl | බියගම | 23 | 202 | 34 | 0 | 0 | 0 | 0 | 0 | 236 | 32 | ගැහැනු අපි එක මිටට....බියගම ආසනයේ මාකොළ  උතුර  පහළ NPP කාන්තා සභාව පිහිටුවීම | 2024-02-05T11:37:00

shaikhsajid1111 commented 7 months ago

I assume that's content part of data. I have not tested this with Sinhala language so yeah it might be something related to encoding. You can tweak and check if something work for you by editing this line. I will look into this issue in my availability