search

lschmelzeisen / nasty

NASTY Advanced Search Tweet Yielder
Apache License 2.0
49 stars 9 forks source link

Problem retrieving tweets which contain emojis only #16

Open ana-sofia93 opened 3 years ago

ana-sofia93 commented 3 years ago

When inspecting the tweets retrieved with the nasty search command, I encountered the following problem:

If a tweet contains only emojis but no text, it cannot be crawled.

Example tweets: https://twitter.com/McDonalds/status/1258532072634724354 https://twitter.com/McDonalds/status/1258894470055149572

Exemplary code for the command line: nasty search --query “(from:@McDonalds)” --max-tweets -1 --since 2020-05-08 --until 2020-05-10 --filter LATEST > mcs.json

It would be possible that my parser does not work as intended, although I have no problems retrieving tweets which include text + emoticons / emojis. However, it might also be a bug in the nasty tool?

lschmelzeisen commented 3 years ago

Yes, this seems to be a bug. I just managed to replicate the problem on my end (also with a more recent McDonalds emoji only Tweet). Thanks for reporting this.

However, at the moment I have no idea what might cause this and can't give you an estimate until when I will manage to fix this. Also, I am fairly confident that I managed to retrieve emoji only Tweets in the past. Weird.

Could be related to #4.