New Feature: Video track on map

[aleksandrmetik]

Hello Android write GPS data layer to MP4 files. We can extract it to GPX format. For example: exiftool -p gpx.fmt FILE.mp4 >out.gpx

A little bit of my thinking about logic:

• If the GPS points in a 1 minute video differ by more than 3 km at any of the video intervals, then we need to "normalize" the track. We must weed out the positioning "noise" from the GPX track, remove segments that differ from neighboring intervals by more than 3 km.
• If video is shorter than 5 minutes and difference between GPS points in the beginning, middle and end of the file does not differ by more than 1 km, then place video icon by starting point.
• If video is shorter than 5 minutes and difference between GPS points in the beginning, middle and end of the file differs more than 1 km - display video icon on the map as GPX track (line).
• If many GPS points are recorded in one minute, for example, more than 30, it will be correct to calculate the average point and record one in a 1-minute interval. This will also minimize GPS noise.

These are my approximate considerations. I'm not sure it's perfect logic. We should definitely consider the need to normalize GPS noise. According to my observations, the noise is especially strong in the first 1-2 minutes of the video, while the GPS "warms up" and less accurate A-GPS is used.

A little bit of considerations about functionality:

• Start GPS point of video should be a average GPS point for first 1-2 minutes of video. This will also minimize GPS noise.
• Display section "My videos", with the ability to turn off and on display (similar to the photo)
• Change settings: display long videos as GPX tracks, or as icons by "video start point". (necessary to perform normalization of GPS noise)
• Edit icons location to display as icons, and edit points when displaying as track.

Thanks!

[aleksandrmetik]

I have implemented a PoC to make sure that it can work. My PoC is a bash script that:

• reads mp4 files from folder;
• finds the GPS data in the mp4;
• extracts the data from them as a track and saves it to track file;
• then with ffpmeg it gets the frames for every 1minute of video (not always good, video has a lot of blurry frames) and with the help of metadata tools - adds to the photo GPS coordinates from the track for that moment;

When scanning through NC/Maps we get GPS tracks as well as a photo on the map.

ToDo:

• Logic to remove GPS noise
• Logic to select a best frame from video (blur, illumination)
• Lots of smaller problems.

Unfortunately, I don't have much time, but maybe this summer I will finish my work and upload a stable MVP version of the script to my repository. Thanks

[executed]

@aleksandrmetik hope you had a great summer back in 2022 as well as the following ones. Do you happen to have any updates on this?

[aleksandrmetik]

Hi @executed Before developing this idea, I asked all my friends to record and send short videos in the original format/source. Unfortunately, 99% of Android devices does not record GPS layer metadata in MP4. There are dedicated Android camera apps that can do this, but agree that that potential usefulness for the vast majority of NextCloud users tends to zero. Maybe in the future when big vendors like Google decide to globally add GPS XMP metadata, or develop a new format for storing video data and GPS, but obviously now this idea from my side is premature.

[aleksandrmetik]

This could be interesting if users manually “put” the video to the Nextcloud map as a single picture, or a hand-drawn track. It can be done not only for MP4/MOV files. I think it should be realized on NC backend side. This would be useful for organizing video files like as photo files.

[executed]

Oh wow, did not realize that Android does not store coordinates... Thanks for the detailed response!