Stickers are sent without information

[z411]

Describe the bug

m.sticker events are sent without image information (like mimetype and thumb size); this results in inconsistent image sizes, animated stickers not playing, and bridges don't handling them proper.

To Reproduce

1. Send a sticker from Nheko
2. Send a sticker from another client that supports sending them like Cinny
3. Compare the raw message

What happened?

Nheko sends this:

{
    "content": {
        "body": "hasan",
        "info": {
            "h": 0,
            "mimetype": "",
            "size": 0,
            "thumbnail_info": {
                "h": 0,
                "mimetype": "",
                "size": 0,
                "w": 0
            },
            "thumbnail_url": "mxc://redacted",
            "w": 0
        },
        "url": "mxc://redacted"
    },
    "event_id": "redacted",
    "origin_server_ts": redacted,
    "sender": "redacted",
    "type": "m.sticker",
    "unsigned": {
        "transaction_id": "redacted"
    }
}

As you can see, ["content"]["info"] has no usable data.

Expected behavior

The same (animated) sticker sent from Cinny looks like this:

{
    "content": {
        "body": "hasan",
        "info": {
            "h": 112,
            "mimetype": "image/gif",
            "size": 37770,
            "thumbnail_info": {
                "h": 112,
                "mimetype": "image/gif",
                "size": 37770,
                "w": 112
            },
            "thumbnail_url": "mxc://redacted",
            "w": 112
        },
        "url": "mxc://redacted"
    },
    "event_id": "redacted",
    "origin_server_ts": redacted,
    "sender": "redacted",
    "type": "m.sticker",
    "unsigned": null
}

As you can see, the info object is filled with appropiate values which makes the sticker looks actually sticker size, properly animated and handled properly by bridge bots.

Screenshots

No response

Version

0.10.2

Operating system

Linux

Installation method

Some repository (AUR, homebrew, distribution repository, PPA, etc)

Qt version

No response

C++ compiler

No response

Desktop Environment

KDE

Did you use profiles?

• [ ] Profiles used?

[deepbluev7]

Can you show the sticker state event, where you got this sticker from? Nheko takes its information from there, so if that has no info field, then Nheko will send none.

[z411]

You're right. Thank you for the clarification

{
    "content": {
        "images": {
            "doremi1": {
                "info": {
                    "h": 169,
                    "mimetype": "image/jpeg",
                    "size": 55289,
                    "w": 223
                },
                "url": "mxc://redacted"
            },
            "testa": {
                "url": "mxc://redacted",
                "usage": [
                    "sticker",
                    "emoticon"
                ]
            }
        },
        "pack": {
            "avatar_url": "mxc://redacted"
        }
    },
    "event_id": "$WTifGgYF_-o5F9VFefL-659la8SYGr5bYwq5LrtSm2g",
    "origin_server_ts": 1672572190528,
    "sender": "redacted",
    "state_key": "",
    "type": "im.ponies.room_emotes",
    "unsigned": {
        "age": 164,
        "prev_sender": "redacted",
        "replaces_state": "$8dRTFCCUUNOKSIQbQIA50hDiKiPYYw4KqomY5JdufMA"
    }
}

This is the sticker state event. "doremi1" was a sticker created by Nheko and "testa" was created by Cinny and Element Experimental.

The weird thing is that Cinny sends the proper image info when sending the m.sticker even if the image info is not defined in the im.ponies.room_emotes event. How does Cinny even know the image information to send in m.sticker if it's not defined in the "info" object?

According to MSC2545, the 'info" object is optional in the image pack event. Should this be considered a Nheko bug (as should Nheko handle stickers without a defined info object) or should I take this to Cinny to properly define their "info" object when adding stickers?

I guess these discrepancies come from using a non-yet specced MSC.

[deepbluev7]

It might make sense to bring it up on the MSC and maybe also give the Cinny devs a friendly poke. Nheko generally doesn't generate the info object for stickers but uses the one from the definition, so that it doesn't need to download the full quality image (in theory). I feel like either the MSC should make it required for stickers or should remove it. However I think the info object is optional for the actual sticker event as well, so that might be why it is optional in the MSC.

I think the experience would be better, if Cinny devs added the info object when uploading the sticker, but maybe Nheko should still try to generate it, if not found.

[JeanPaulLucien]

I see expected behavior. Windows 7.

[z411]

Was the sticker created in Nheko or Cinny? This happens specifically when the sticker is created in Cinny and sent from Nheko, at least for us.

[deepbluev7]

@JeanPaulLucien It is a problem with Cinny <-> Nheko interaction and we already found the root cause. Which behaviour we want is still to be determined.

[JeanPaulLucien]

I sent sticker from Nheko and checked an information by Nheko. It was okay. Yeah, I did not read about Cinny.