search

sebinsua / scrape-twitter

🐦 Access Twitter data without an API key. [DEPRECATED]
GNU General Public License v3.0
176 stars 36 forks source link

Parser stripping suffix from number #15

Closed mikebull closed 6 years ago

mikebull commented 6 years ago

I've noticed that when I scrape a popular Twitter profile with over ten thousand tweets/followers/following/etc, the numbers I get back from the scraper are usually much smaller than they should be.

After a bit of digging, it looks like the counts in parser.js are taking the suffixed value (i.e. 15.5k) and is stripping out non-numerical characters, so 155 is returned instead of 15500.

Instead of grabbing the suffixed number, do you see any issues with grabbing the data-count attribute, and using that instead? If not, I'll create a PR and send it your way.

diego1199 commented 6 years ago

Hi @mikebull , can you share your solution for the parser?

mikebull commented 6 years ago

@diego1199 I've made the necessary change on a fork of this repo.

I've also added a PR, where you can see the change I made.