knadh / tg-archive

A tool for exporting Telegram group chats into static websites like mailing list archives.
MIT License
903 stars 139 forks source link

--sync doesn't download animated stickers #114

Open scarlion1 opened 1 year ago

scarlion1 commented 1 year ago

When --sync comes across a message with an animated sticker, looks like it just saves the emoji associated with it.  Unfortunately I'm not a coder or anything, so not sure how to debug this more, but I was able to look in the tg-archive database.

Animated Sticker Message

1. media table has no entry

2. messages table entry:

id type date edit_date content reply_to user_id media_id
123 message {earlier}   🤑   98765  

3. @RawDataBot info:

{
    "update_id": X,
    "message": {
        "message_id": X,
        "from": {
            me
        },
        "chat": {
            my private
        },
        "date": X,
        "forward_from": {
            me
        },
        "forward_date": {earlier},
        "sticker": {
            "width": 512,
            "height": 512,
            "emoji": "\ud83e\udd11",
            "set_name": "KangarooFighter",
            "is_animated": true,
            "is_video": false,
            "type": "regular",
            "thumbnail": {
                "file_id": "AAMCAgADGQEAASYIzWUGY0H7SOvgHiO5_grFOqqhuTzzAAIpAANZu_wlaIR2LDtJZnsBAAdtAAMwBA",
                "file_unique_id": "AQADKQADWbv8JXI",
                "file_size": 6868,
                "width": 128,
                "height": 128
            },
            "thumb": {
                "file_id": "AAMCAgADGQEAASYIzWUGY0H7SOvgHiO5_grFOqqhuTzzAAIpAANZu_wlaIR2LDtJZnsBAAdtAAMwBA",
                "file_unique_id": "AQADKQADWbv8JXI",
                "file_size": 6868,
                "width": 128,
                "height": 128
            },
            "file_id": "CAACAgIAAxkBAAEmCM1lBmNB-0jr4B4juf4KxTqqobk88wACKQADWbv8JWiEdiw7SWZ7MAQ",
            "file_unique_id": "AgADKQADWbv8JQ",
            "file_size": 65479
        }
    }
}


For comparison:

Non-Animated Sticker Message

1. media table entry:

id type url title description thumb
234 photo 234.webp sticker.webp    

2. messages table entry:

id type date edit_date content reply_to user_id media_id
234 message {afterwards}       87654 234

3. @RawDataBot info:

{
    "update_id": X,
    "message": {
        "message_id": X,
        "from": {
            me
        },
        "chat": {
            my private
        },
        "date": X,
        "forward_sender_name": "someone else",
        "forward_date": {recent},
        "sticker": {
            "width": 512,
            "height": 512,
            "emoji": "\ud83d\ude08",
            "set_name": "moshfox",
            "is_animated": false,
            "is_video": false,
            "type": "regular",
            "thumbnail": {
                "file_id": "AAMCAQADGQEAASYJC2UGg5kGxKboLtnJOqxRTyGCD9aAAAKYAQACZ93RRWNTU8i_bhvwAQAHbQADMAQ",
                "file_unique_id": "AQADmAEAAmfd0UVy",
                "file_size": 9110,
                "width": 320,
                "height": 320
            },
            "thumb": {
                "file_id": "AAMCAQADGQEAASYJC2UGg5kGxKboLtnJOqxRTyGCD9aAAAKYAQACZ93RRWNTU8i_bhvwAQAHbQADMAQ",
                "file_unique_id": "AQADmAEAAmfd0UVy",
                "file_size": 9110,
                "width": 320,
                "height": 320
            },
            "file_id": "CAACAgEAAxkBAAEmCQtlBoOZBsSm6C7ZyTqsUU8hgg_WgAACmAEAAmfd0UVjU1PIv24b8DAE",
            "file_unique_id": "AgADmAEAAmfd0UU",
            "file_size": 15866
        }
    }
}


I downloaded the animated sticker message via the desktop client, AnimatedSticker.tgs is the file name, zipped up and attached here in case that helps.  AnimatedSticker.tgs.zip

The primary goal here is to at least get the AnimatedSticker.tgs downloaded with the rest of the media and show a hyperlink to the file in the message with the emoji.  Ideally, it could be displayed inline but I don't think browsers even support TGS animations.  Until then, would need some sort of costly conversion to animated webp I guess?

I also noticed the content in the messages table for a regular sticker is empty but I believe the emoji should be saved there, yes?  Perhaps that can be addressed as well.

Hope that's helpful.  Here's Telegram's info about stickers: https://core.telegram.org/stickers

scarlion1 commented 1 year ago

Now I'm seeing some animated stickers do get downloaded as a sticker.webm media file...

#### [@RawDataBot](https://t.me/RawDataBot) info: ``` { "update_id": X, "message": { "message_id": X, "from": { me }, "chat": { my private }, "date": X, "forward_from": { someone }, "forward_date": X, "sticker": { "width": 512, "height": 512, "emoji": "\ud83c\udf89", "set_name": "SiameseCatLive2", "is_animated": false, "is_video": true, "type": "regular", "thumbnail": { "file_id": "AAMCAQADGQEAASc2dWU-1VJZTqkdNB1wzssfMWj9SQo2AALkAgACTrQwRAnrC6zA9mKYAQAHbQADMAQ", "file_unique_id": "AQAD5AIAAk60MERy", "file_size": 15858, "width": 320, "height": 320 }, "thumb": { "file_id": "AAMCAQADGQEAASc2dWU-1VJZTqkdNB1wzssfMWj9SQo2AALkAgACTrQwRAnrC6zA9mKYAQAHbQADMAQ", "file_unique_id": "AQAD5AIAAk60MERy", "file_size": 15858, "width": 320, "height": 320 }, "file_id": "CAACAgEAAxkBAAEnNnVlPtVSWU6pHTQdcM7LHzFo_UkKNgAC5AIAAk60MEQJ6wuswPZimDAE", "file_unique_id": "AgAD5AIAAk60MEQ", "file_size": 109117 } } } ```

The difference seems to be this one is marked "is_animated": false and "is_video": true whereas the one from OP that gets skipped is marked "is_animated": true and "is_video": false.  I'm not sure how/where this is handled in the code but maybe just need to add a check for is_animated and treat it the same as when is_video == true?  If you can point me to the relevant place in the code I can do some testing...