Google Motion Photos

[plodocus]

Describe the solution you'd like Google Pixel phones can take so called "motion photos", which are like iOS live photos (see #344). I'd like photos to be able to show the video within the picture. In the official Google Photos app there is a button that toggles display of picture or video.

Additional context These photos are JPGs with embedded MP4 videos: https://stackoverflow.com/questions/53104989/how-to-extract-the-photo-video-component-of-a-mvimg Since the Pixel camera app stores them with the prefix "MVIMG" only the files matching this could be scanned for videos. Also see https://github.com/cliveontoast/GoMoPho

[MaxG-Git]

Would love to see this implimented! 😄

In the mean time if you have already extracted your files not using the extractor listed above and need to change the filetype to mp4, a clucky method can be found here

[SimplyCorbett]

Would also love to see this!

[Derkades]

Since the Pixel camera app stores them with the prefix "MVIMG"

The image name format has changed, PXL_date_time.jpg for regular photos and PXL_date_time.MP.jpg for motion pictures

[skjnldsv]

Couldn't find a javascript api for this. Google photos seems to be using the backend to serve img/video, they don't just do it from the browser! :confused: :thinking:

[cooperra]

I'd like to work on this.

My current plan is to extend nextcloud/viewer front-end to handle micro videos contained in JPEGs that match Google's naming conventions (MVIMG.jpg and .MP.jpg). That way we avoid downloading full-sized versions of probably motionless JPEGs.

When the file name matches, we'll display a toggle button. When the button is active, we'll fetch the full-sized file (containing both the offset metadata and video data). If we have metadata, we should be able to put that video data into a player and handle it like any other video. If the metadata is missing, we revert to still-image behavior and hide the button.

In a future update, we could find or build a way to fetch the EXIF metadata without downloading the whole file. Then, we could check the metadata directly instead of checking the filename.

[skjnldsv]

@cooperra exactly! That seems like a clear and neat approach! Unfortunately I don't think there is a way to easily extract the video from the MVIMG in javascript. Do you have any idea how to approach this?

[Derkades]

Couldn't you, in javascript, download the file and split it by looking for header bytes for the image and video? I'm not a javascript developer and I don't know if you have byte level access to files like this.

https://github.com/cliveontoast/GoMoPho/blob/master/src/GoMoPhoConsole/Program.cs https://github.com/joemck/ExtractMotionPhotos/blob/v2.0/main.c

[cooperra]

This is the best resource I've found on the topic: https://android.jlelse.eu/working-with-motion-photos-da0aa49b50c

We don't need to search for header bytes. The EXIF data contains a MicroVideoOffset value that points to the start of the video data. We just need to download and parse the EXIF data, then carve the video out of the MVIMG file.

[Exilit]

My current plan is to extend nextcloud/viewer front-end to handle micro videos contained in JPEGs that match Google's naming conventions (MVIMG.jpg and .MP.jpg). That way we avoid downloading full-sized versions of probably motionless JPEGs.

Sounds good. As far as I can see, the viewer is only responsible for the full size view of the images, right? Especially for ios Live photos (#344 ) I think there is another thing we should pay atention to. Apples Live Photos do store two seperate files (JPEG/HEIC and a .MOV file) resulting in the preview or album view being cluttered with duplicate images.

A possible solution could be not to show a .MOV file if there is an image file with the same name available. To make sure we are really hiding a video from an Live Photo we could also check for the uuid (https://stackoverflow.com/questions/32508375/apple-live-photo-file-format).

[skjnldsv]

A possible solution could be not to show a .MOV file if there is an image file with the same name available.

People working on videos might store the preview alongside the video, with the same title. Even if it's not a live photo. We actually provide this featuyre for videos. We show any same-name image as poster of the video when it ends.

So it's too many edge cases to determine whether to show or not the same-name files. Thus we show everything. It's too uch of work and not worth the trouble :)

In any wase, that is not related to this issue, which is about google motion photos :rocket:

[Exilit]

I am sorry for being off-topic. I was thinking about contributing with the Live Photo support and thought it might be worth to talk about here.

So it's too many edge cases to determine whether to show or not the same-name files.

Well, to be completely sure you could also compare the assetIdentifier like I stated above. It would also be possible to make this configurable so that users can choose.

In the end I think there will be a way to handle that. Other cloud Services like Google Photos do this already pretty well and I would really love to see this in NC aswell.

Thus we show everything. It's too uch of work and not worth the trouble :)

So do you think it’s not worth it investigating further thoughts into this?

[chrirauch]

@cooperra exactly! That seems like a clear and neat approach! Unfortunately I don't think there is a way to easily extract the video from the MVIMG in javascript. Do you have any idea how to approach this?

@Derkades Not a drop in script, but this should probably give you an idea...

https://github.com/dj0001/Motion-Photo-Viewer

[lbdroid]

I think there may be some subtle differences between MVIMG and MP.jpg versions. Some messages above suggest there to be an exif tag "MicroVideoOffset" for MVIMG. There is no such tag for MP.jpg. Instead, there is "Directory Item Length", but the catch is that this tag appears twice. Here's an example:

$ exiftool -a somefile.MP.jpg | grep "Directory Item"
Directory Item Mime             : image/jpeg
Directory Item Semantic         : Primary
Directory Item Length           : 0
Directory Item Padding          : 0
Directory Item Mime             : video/mp4
Directory Item Semantic         : MotionPhoto
Directory Item Length           : 2518994
Directory Item Padding          : 0

Obviously we need the second one and not the first one.

Now the nice thing is that these 2518994 bytes are from the end of the file, so we can calculate the offset easily by subtracting that from the total file length and adding 1.

Here's a quick and cludgy bash script to take out the mp4 from the combined file;

#!/bin/bash

VLEN=`exiftool -a $1 | grep MotionPhoto -A1 | grep Length | cut -d\: -f2 | sed -e 's/^[ \t]*//'`
FLEN=`ls -l $1 | cut -d\  -f5`
START=$(($FLEN-$VLEN))
dd bs=$START skip=1 if=$1 of=`echo $1 | sed -e "s/jpg/mp4/"`

Just feed in the filename as the first and only parameter and out pops the mp4.

[Gurkengewuerz]

Since the announcement of Nextcloud Hub 3 with the recognition app i am amazed to migrate my Photo Sync completly to Nextcloud instead of Google Photos. Since i am a Pixel User i have a lot of Motion Photos so i would love to see this feature.

[darkneo29]

Same. I just migrated to nextcloud. Motion photos would be amazing

[TacoCake]

Since I would also greatly enjoy being able to view Motion Photos from the 'Photos' app, I think it would be useful to mention that Samsung's Motion Photos are also videos embedded into JPG files.

However, in Samsung's case (at least as of Android 13), there is no filename difference between motion photos and standard photos.

[StampixSMO]

Since I would also greatly enjoy being able to view Motion Photos from the 'Photos' app, I think it would be useful to mention that Samsung's Motion Photos are also videos embedded into JPG files.

However, in Samsung's case (at least as of Android 13), there is no filename difference between motion photos and standard photos.

Samsung uses JPEG marker 0x60 to store the MPEG video, so that could be checked to determine presence of a Motion Photo.

[cromefire]

As fast as I'm aware the easiest solution would be to just put the photo into a <video> tag and override it's mine type, right (then it'll search the MP4 header)? It's a bit wasteful in downloading everything, but at least it should easily work. For now an identification via the PXL_*.MP.jpg also seems sufficient or you could make this an opt-in feature for now.

[cooperra]

I don't think the motion part is too bad. The tricky part is the the UI to toggle it. I don't think the viewer was designed with alternative viewing modes in mind. Items are expected to be videos or photos, but not both. This distinction is made server-side. The problem I ran into was that I couldn't check for motion metadata in a photo without downloading the whole file to the client. That would be too expensive to do for every photo when only a few have motion. I suppose the filename heuristic could work in the short term, but ideally this would be flagged server-side. Maybe in preview generation? I didn't want to open that can of worms, but maybe someone with more motivation could pull it off.

In the end, I switched to the Memories app. They already have support for this there.