ferguras / twitter-analysis

Scrape the Twitter Frontend API without authentication.
MIT License
25 stars 4 forks source link

Fix common encoding problems with ftfy #5

Closed briwa closed 5 years ago

briwa commented 5 years ago

Hello, thanks for maintaining the code from the previous owner. Just stumbled upon some problems with the tweet body text. Some characters get unexpectedly encoded (For example, '”' would turn into \u201c). This could've been fixed by ftfy. At least this would fix most of the common encoding issue. What do you think?

Thanks.