HTML-formatted Media Captions not supported by Element-IOS client

RannerDesign commented 4 weeks ago

Steps to reproduce

HTML-formatted Media Captions not supported by Element-IOS client

The Client-Server-API describes in chapter 10.2.2.7 how to set up captions for media https://spec.matrix.org/unstable/client-server-api/#media-captions

After solution of Issue 27927 Element/Web and Element/Windows are supporting unformatted and HTML-formatted captions.

IOS and Android clients currently do not support captions.

The tests were performed in the following environments:

Element Desktop on Windows 10 pro 22H2 (Build 19045.4780) Version von Element: 1.11.83 Krypto-Version: Rust SDK 0.7.2 (7a21514), Vodozemac 0.7.0

Element IOS client Element 1.11.18 (20240917151940) Matrix SDK 0.27.13 Rust Crypto SDK 0.7.2 (Vodozemac 0.7.0)

The following server and user environment was used:

user_name:  Testuser1040
user_id:    @m1040a:matrix.org
room_name:  TH3
room_id:    !mNtkvlOYMJmlSPpMZt:matrix.org

The generation of events was performed with Python scripts using matrix-nio

Results for Element/Windows and Element/Web are now as expected: the image is show with a formatted caption.

Result for Element/IOS is unexpected: the image is shown without caption.

241103 Screenshots The sources for these events are nearly identical:

Element/Windows

{
    "content": {
        "body": "Fotograf 3713: Drehwurm",
        "filename": "WP2024-03_Drehwurm_[Ranner]_IMG_3713.jpg",
        "format": "org.matrix.custom.html",
        "formatted_body": "<p><strong>Fotograf 3713: </strong>Drehwurm</p>",
        "info": {
            "h": 427,
            "mimetype": "image/jpeg",
            "size": 205735,
            "thumbnail_info": null,
            "thumbnail_url": null,
            "w": 640
        },
        "msgtype": "m.image",
        "url": "mxc://matrix.org/OTDzouHVfiTUncRpkUHoitSA"
    },
    "origin_server_ts": 1730324921229,
    "sender": "@e1035z:matrix.org",
    "type": "m.room.message",
    "unsigned": {
        "membership": "join",
        "age": 343618259
    },
    "event_id": "$Tq23xs6YTdrGfxnAeugyGZR_wlq8Du96p3xskytiO7U",
    "room_id": "!mNtkvlOYMJmlSPpMZt:matrix.org"
}

Element/Web

{
    "content": {
        "body": "Fotograf 3713: Drehwurm",
        "filename": "WP2024-03_Drehwurm_[Ranner]_IMG_3713.jpg",
        "format": "org.matrix.custom.html",
        "formatted_body": "<p><strong>Fotograf 3713: </strong>Drehwurm</p>",
        "info": {
            "h": 427,
            "mimetype": "image/jpeg",
            "size": 205735,
            "thumbnail_info": null,
            "thumbnail_url": null,
            "w": 640
        },
        "msgtype": "m.image",
        "url": "mxc://matrix.org/OTDzouHVfiTUncRpkUHoitSA"
    },
    "origin_server_ts": 1730324921229,
    "sender": "@e1035z:matrix.org",
    "type": "m.room.message",
    "unsigned": {
        "membership": "join",
        "age": 343955184
    },
    "event_id": "$Tq23xs6YTdrGfxnAeugyGZR_wlq8Du96p3xskytiO7U",
    "room_id": "!mNtkvlOYMJmlSPpMZt:matrix.org"
}

Element/IOS

{
    "sender": "@e1035z:matrix.org",
    "content": {
        "format": "org.matrix.custom.html",
        "formatted_body": "<p><strong>Fotograf 3713: </strong>Drehwurm</p>",
        "url": "mxc://matrix.org/OTDzouHVfiTUncRpkUHoitSA",
        "msgtype": "m.image",
        "filename": "WP2024-03_Drehwurm_[Ranner]_IMG_3713.jpg",
        "body": "Fotograf 3713: Drehwurm",
        "info": {
            "size": 205735,
            "mimetype": "image/jpeg",
            "w": 640,
            "h": 427
        }
    },
    "origin_server_ts": 1730324921229,
    "room_id": "!mNtkvlOYMJmlSPpMZt:matrix.org",
    "event_id": "$Tq23xs6YTdrGfxnAeugyGZR_wlq8Du96p3xskytiO7U",
    "unsigned": {
        "age": 344442504
    },
    "type": "m.room.message"
}

A support of formatted captions for Element/IOS client like Element/Windows and Element/Web would be highly appreciated.

Outcome

What did you expect?

Image with caption

What happened instead?

Image without caption

Your phone model

iPhone Xs

Operating system version

IOS 18.0.1

Application version

No response

Homeserver

matrrix.org

Will you send logs?

No

sophonet commented 2 days ago

I have seen that the type „defect“ has been changed to „enhancement“, but I would consider this a defect: It is quite common that people send pictures with additional text e.g. using WhatsApp, and the text, even if it is technically called „caption“, often includes relevant information / personal questions etc. to the recipient. Currently according to this bug it is dropped from the conversation. In my situation it has caused confusion a couple of times already since questions in the caption have not been answered.

So indeed I would be glad if this issue could be addressed and the caption is shown as text below a picture, like it is done in WhatsApp/Signal/other messenger clients or in the desktop/web versions of element.

manuroe commented 2 days ago

This feature is happening on ElementX-iOS: https://github.com/element-hq/element-x-ios/pull/3429. The whole feature is available on EXI nightlies. It will be released soon on the App Store.

element-hq / element-ios