lay295 / TwitchDownloader

Twitch VOD/Clip Downloader - Chat Download/Render/Replay
MIT License
2.72k stars 261 forks source link

Fallback to archive site for removed global Twitch emotes #1241

Open superbonaci opened 5 days ago

superbonaci commented 5 days ago

Checklist

Edition

Command Line Interface

Describe your issue here

Older chats got the emote removed even if it hadn't expired back then. So the url for the image is there but can't be downloaded:

$ TwitchDownloaderCLI chatdownload -E -u 2262876737 -o 2262876737.html
$ TwitchDownloaderCLI chatdownload -E -u 2262876737 -o 2262876737.json

This could happen also to other images. If TwitchDownlaoderCLI can't get the images from twitch it can try from Web Archive. This affects chatdownload chatupdate chatrender.

chats: 2262876737.html.zip 2262876737.json.zip

Add any related files or extra information here

2262876737

ScrubN commented 4 days ago

I like the idea of restoring deleted Twitch emotes such as BibleThumb, PogChamp, and BlessRNG, I also don't really want to keep a hard coded list of removed emotes that fallback to archive.org. Maybe I'll change my stance on it later.

superbonaci commented 4 days ago

Also OMGScoots. You could store the images in an assets folder in the project, but I don't know if that is legal. But if you did that you could do it with all emotes, and the program will be much faster.

If you don't hard code the list of removed emotes in the program the only thing you can do is to connect to a website that lists all the removed emotes, and is updated. But I don't know of such website and I don't think it will ever exist.

Using archive.org is simple and fixes the problem now, you can even keep a list of urls for each removed emote in case archive.org closes or something.