search

element-hq / element-meta

Shared/meta documentation and project artefacts for Element clients
66 stars 11 forks source link

Display "Motion Photos" correctly #1344

Open axelsimon opened 5 years ago

axelsimon commented 5 years ago

Problem description Google introduced a new format of photos with the Pixel 2 phone called Motion Photos, which are essentially a photo with a bit of video "around it": the camera will capture up to 1.5 sec of video before and after the photo is taken, and then store this video at the end of the jpeg file.

The file names start with MVIMG.

When displayed, the photos give an effect of what can be best described as a moving photo. Technically a video, but giving the feeling of a photo.

Describe the solution you'd like It would be great if Riot could recognise motion photos and display the video they contain, either directly in the chat (pre)view, or only on click/download. In any case, a clear marker that the picture is a special picture would be useful.

Describe alternatives you've considered The obvious alternative is not to manage MVIMGs.

Additional context https://android.jlelse.eu/working-with-motion-photos-da0aa49b50c

t3chguy commented 5 years ago

do browsers support these magical files? Are there any webapps you know, facebook, twitter etc that support them correctly? I cannot see anything useful for a google search for "HTML5 MVIMG"

axelsimon commented 5 years ago

Upon further inspection of a MVIMG_date_time.jpg that was sent to me via Riot, it appears the mp4 video part at the end of the file is missing, as confirmed by the fact xxd MVIMG_20190410_192832.jpg | grep mp4 returns nothing.

I'm wondering if Android strips the video from the jpeg before giving the picture to Riot to share.

axelsimon commented 5 years ago

@t3chguy Sorry, I don't, but that's a good point. I took the liberty to open this feature request/suggestion as the format appeared both simple enough and not proprietary to be worth considering. I haven't really checked much further.

ara4n commented 5 years ago

i'd be up for this too, as iOS has had Live Photos for years.

The way they are structured is as a combo JPEG + video file, so the way we'd represent them in Matrix would probably be as an extensible event which is video with a JPEG fallback (separate to the thumbnail), with a custom event type so that aware clients could show the right UI.

It'd probably mean the sending client splitting the resource into JPEG + video, but this isn't hard (at least with iOS).