search

matrix-org / synapse

Synapse: Matrix homeserver written in Python/Twisted.
https://matrix-org.github.io/synapse
Apache License 2.0
11.83k stars 2.12k forks source link

HEIC/HEIF support #13767

Open ptman opened 2 years ago

ptman commented 2 years ago

Description:

Apparently someone managed to send an HEIC image from an apple device to a matrix room I participate in. At least thumbnailing HEIC/HEIF ( https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format ) would be nice. I suppose synapse isn't supposed to be a generic media converter, but that would have been useful as well.

PNGs should be thumbnailed as JPEG https://github.com/matrix-org/synapse/issues/2045 , but there's complaints about WEBP being thumbnailed about JPEG https://github.com/matrix-org/synapse/issues/11840 . I guess it's hard to please everyone.

ACiDGRiM commented 1 year ago

I experience this issue as well, it's also inconsistent which devices can view the HEIC. Samsung phones can view them, a Windows 10 computer with the required HEIC/HEVC modern applications, however other windows 10/11 computers can't even with the windows store addons.

I agree with ptman, it would be nice if an external library at least could be called to thumbnail them for clients that don't support this format.

olmari commented 1 year ago

AFAIK as an basic thing thumbnailer should at least thumbnail things on the original format, if receiver can't show it then there is not much we can do... Sure alternatively specs could mandate which types could be used withing thumbnails and convert away where needed, but that is always set of own issues...

Currently in our server we see an sent HEIC-image tried to be thumbnailed as png by media-worker, and failing at that (well, HEIC isn't PNG, but apparently it treats it as it is).

ADD: Apparently thumbnailer fails when Element-desktop "asks" thumbnail (tries to do png?):

2023-07-01 23:14:13,367 - synapse.media.media_repository - 583 - WARNING - GET-3065 - Unable to generate a thumbnail for local media lQsnzkTjYcUVxMNOKuLbAGHm using a method of scale and type of image/png: 
2023-07-01 23:14:13,367 - synapse.rest.media.thumbnail_resource - 199 - WARNING - GET-3065 - Failed to generate thumbnail

Whereas Element android shows the image in chat normally (be it thumbnailed or the full original in the screen).

ACiDGRiM commented 1 year ago

Whereas Element android shows the image in chat normally (be it thumbnailed or the full original in the screen).

Just an observation, but it seems android only displays the preview on more recent phones. One user had a Samsung s7 and couldn't see the thumbnails, but users with S8 series and later are able to. Once the user ypgraded to an s10, he could see the thumbnails. Users on windows 10 with the heif windows store add on are able to see the thumbnails in some cases

But Linux users can't see thumbnails and have to download the blank preview to see in an alternate application