search

JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.48k stars 712 forks source link

TwitterSearchScraper doesn't include retweets when searched for a keyword or hashtag #285

Closed raza8899 closed 3 years ago

raza8899 commented 3 years ago

I am using TwitterSearchScraper to search based on a keyword or hashtag but the result set doesn't include the retweets for a given keyword. Is there a way to add include:retweets in the query parameter. Please help

raza8899 commented 3 years ago

e.g The Twitter API gives the following result:

Date Text User
Fri Apr 10 20:22:12 +0000 2020 RT @LogisticsStuff: VIDEO: The Tesla Semi Will... LearnShareBuild
Fri Apr 10 19:05:08 +0000 2020 Supervised Self-Driving Shuttles Move COVID-19... acmTechNews
Fri Apr 10 19:00:33 +0000 2020 Create a #SelfDrivingCar to roll on its own fo... ElectromakerIO
Fri Apr 10 18:50:47 +0000 2020 RT @CapgeminiAIE: #FridayReads\nInnovations in... zensarkar
Fri Apr 10 17:33:55 +0000 2020 RT @AIinHealthcare: ICYMI: #MayoClinic CEO: “U... soychemist
Fri Apr 10 17:15:04 +0000 2020 ICYMI: #MayoClinic CEO: “Using #AI enables us ... AIinHealthcare
Fri Apr 10 11:12:18 +0000 2020 With the modular E-Flex platform, up to 22 dif... Inceptive_Mind
Fri Apr 10 10:02:06 +0000 2020 RT @ElectromakerIO: Check out all of our neat ... JacobusSystems
Fri Apr 10 10:00:24 +0000 2020 Check out all of our neat @NXP Cup entries so ... ElectromakerIO
Fri Apr 10 09:44:27 +0000 2020 RT @CapgeminiAIE: #FridayReads\nInnovations in... iot_automotive
Fri Apr 10 06:50:50 +0000 2020 RT @CapgeminiAIE: #FridayReads\nInnovations in... LannySCohen
Fri Apr 10 05:43:51 +0000 2020 RT @CapgeminiAIE: #FridayReads\nInnovations in... raveenabeniwal
Fri Apr 10 05:05:45 +0000 2020 #FridayReads\nInnovations in #AV may transform... CapgeminiAIE
Fri Apr 10 02:19:05 +0000 2020 RT @sujamthe: My Students inspire me! 🚗 Wonder... CoachCaroleM

Whereas the snscrape gives the following results

2020-04-10 19:05:08+00:00   1248688483318878208 Supervised Self-Driving Shuttles Move COVID-19...   acmTechNews
2020-04-10 19:00:33+00:00   1248687329918795776 Create a #SelfDrivingCar to roll on its own fo...   ElectromakerIO
2020-04-10 17:15:04+00:00   1248660785754255360 ICYMI: #MayoClinic CEO: “Using #AI enables us ...   AIinHealthcare
2020-04-10 11:12:18+00:00   1248569489475567617 With the modular E-Flex platform, up to 22 dif...   Inceptive_Mind
2020-04-10 10:00:24+00:00   1248551397286649857 Check out all of our neat @NXP Cup entries so ...   ElectromakerIO
2020-04-10 05:05:45+00:00   1248477246676819969 #FridayReads\nInnovations in #AV may transform...   CapgeminiAIE
raza8899 commented 3 years ago

@JustAnotherArchivist So basically as you can see above the retweets made that day aren't returned, rest all is returned. Please let me know if there is a way through which retweets are also enabled.

JustAnotherArchivist commented 3 years ago

That's correct. This is a limitation of the Twitter search page, and there is no workaround that works beyond the past 7 days.

Duplicate of #96