search

JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.33k stars 699 forks source link

Tweet replies #98

Closed hayn5126 closed 3 years ago

hayn5126 commented 3 years ago

Is it possible to get all replies for each tweet?

The following code only gives the tweets but not the reply for this tweet (under this tweet). snscrape --jsonl twitter-search "from:barackobama since:2015-09-10 until:2015-09-12" > baracktweets.txt

I tried to use twitter-thread with this tweet https://twitter.com/selenagomez/status/1291079043333459976 snscrape --jsonl twitter-thread "1291079043333459976" But I get an error:

2020-10-02 08:42:16.180  WARNING  snscrape.modules.twitter  Tweet does not exist, is not a thread, or does not have ancestors

I've tried the conversation id also, still couldn't get it to work. Any help would be appreciated. Thank you

Also, is it correct that snscrape twitter-search "from:barackobama" is the same as snscrape twitter-user "barackobama"

They seem to be giving me the same results.

JustAnotherArchivist commented 3 years ago

Scraping replies is currently not supported. See #51.

I found yesterday that twitter-thread is broken entirely. I forgot to update it when I added various extra fields, but also it's still based on the old Twitter design and hard to maintain. I'll likely rewrite that from scratch soon. Note that when it still worked, twitter-thread was going backwards in a thread; i.e. you'd give it some tweet, and it'd fetch the ancestors up to the start of the thread. So the tweet you linked never worked with it because it is not a reply to some other tweet.

Yes, twitter-user is a wrapper around twitter-search using from:. The only functional difference between the two is the entity (--with-entity), which provides some info on the user extracted from the profile page and does not exist on twitter-search.

Fatima-Haouari commented 3 years ago

Hi, Could you please

Scraping replies is currently not supported. See #51.

I found yesterday that twitter-thread is broken entirely. I forgot to update it when I added various extra fields, but also it's still based on the old Twitter design and hard to maintain. I'll likely rewrite that from scratch soon. Note that when it still worked, twitter-thread was going backwards in a thread; i.e. you'd give it some tweet, and it'd fetch the ancestors up to the start of the thread. So the tweet you linked never worked with it because it is not a reply to some other tweet.

Yes, twitter-user is a wrapper around twitter-search using from:. The only functional difference between the two is the entity (--with-entity), which provides some info on the user extracted from the profile page and does not exist on twitter-search.

Hi, Is there any plan for updating the code to get the replies soon. I appreciate you help.

Fatima-Haouari commented 3 years ago

Scraping replies is currently not supported. See #51.

I found yesterday that twitter-thread is broken entirely. I forgot to update it when I added various extra fields, but also it's still based on the old Twitter design and hard to maintain. I'll likely rewrite that from scratch soon. Note that when it still worked, twitter-thread was going backwards in a thread; i.e. you'd give it some tweet, and it'd fetch the ancestors up to the start of the thread. So the tweet you linked never worked with it because it is not a reply to some other tweet.

Yes, twitter-user is a wrapper around twitter-search using from:. The only functional difference between the two is the entity (--with-entity), which provides some info on the user extracted from the profile page and does not exist on twitter-search. Hi, Thanks for your efforts. Any plans to implement this soon. I appreciate your help.