Twitter Ripper Skips Large Numbers of Files - Githubissues

RipMeApp / ripme

Downloads albums in bulk

MIT License

3.73k stars 630 forks source link

Twitter Ripper Skips Large Numbers of Files #1674

Open flaccidbagel opened 4 years ago

flaccidbagel commented 4 years ago

Ripme version: 1.7.92
Java version: Version 8 Update 251
Operating system: Windows 10
Exact URL you were trying to rip when the problem occurred: (NSFW) https://twitter.com/negi_futa
Please include any additional information about how to reproduce the problem: Attempt downloading from any Twitter user

Expected Behavior

RipMe rips all images/videos

Actual Behavior

Most images fail to download. While ripping, it shows no direct errors for the images it skips, nor shows any attempt at downloading them. Log file attached shows the following message:

2020-05-28 21:11:56,840 ERROR ripper.AbstractRipper .getURLsFromJSON() XXX Tweet doesn't have entities

log.txt rip.properties.txt

literaldehyde commented 4 years ago

Same here, effectively broken for twitter.

Isaaku commented 4 years ago

Are you counting the retweets? This message (XXX Tweet doesn't have entities) should only appear when the tweet is only text

flaccidbagel commented 4 years ago

Many of the missing elements aren't retweets. Unfortunately, not really finding any trends between what does get missed and what does not.

bonnetduforme commented 4 years ago

Same here, and I have a theory. I noticed that the account I want to rip all images removed some of them. Since Twitter can still have them in memory, maybe RipMe stops working when confronted with one of them?