correction of gpx data (bad gps data in tunnels and other cases)

[vilmic1]

As we know, GPS doesn't work in tunnels. GOPRO generates a lot of false values. To adjust them, I export GPS data from GOPRO files using gopro-to-gpx.py. Later, I adjust them (coordinates and altitude) and submit the gpx file (together with the original GOPRO video file) to gopro-dashboard.py. Coordinates, altitude, maps are displayed perfectly, but I don't understand something with the speed display. Sometimes the speed is what I calculated from the gpx data, sometimes it goes up or down a lot regardless of the gpx file I submitted. It seems that not only the data from the adjusted gpx file is involved in the speed calculation. How is this really happening, is gopro-dashboard.py sometime ignoring the gpx file I submitted or are there errors in my file? For now, I would like to figure out the speed calculation algorithm, and then look for errors on my own.

If anyone is interested, I will describe my algorithm for adjusting GPS coordinates. 1) I create a video with original (false) GPS data. In it, I can see the timestamp and position on the map, when the coordinates go off track and when they get back on track, and when they return to normal speed. I record the timestamps of these events. They will be needed to adjust the gpx file. 2) I export GPS data using gopro-to-gpx.py. The gpx file can be viewed on the website https://www.gpsvisualizer.com. 3) I submit the gpx file to the program I created for correction. I don't program in py, so I haven't looked at the source code or modified it. I wrote the app in go (golang). I give her an exported gpx file and an xml file in a very similar format (gpx) that lists the adjustment rules (many of them for each adjustment event). I give start and end timestamps for each event (just like gpx). Between these marks, the original coordinates and altitude are ignored and recalculated according to the straight line equation (for now it is like this, maybe I will improve it). All points in the original gpx file are recalculated, so the number of points in the gpx file does not change and the timestamp of each point does not change. If the road (tunnel) is not straight, then I can put intermediate coordinates that I get from google maps. I get a polygon of lines (I'll use a spline to smooth the lines for now). When describing waypoints, I can specify the timestamp (the timestamp of the waypoint of the correction rule, not the timestamp in the result file) directly (this will depend on the speed in the specified section), or specify that the timestamp be calculated automatically based on the distance between the coordinates, estimating that the speed during the entire event is constant (other options are possible, so far it's done this way). 4) I get an adjusted gpx file from my app that I can view at https://www.gpsvisualizer.com. and if necessary, repeat the process several times until I get the desired result. 5) I submit the adjusted gpx file (together with the original GOPRO video file) to gopro-dashboard.py. The number of points and the time stamp of each point correspond to the data of the original gpx file, only the coordinates and altitude have been changed. The speed in the adjustment events should be constant, but I sometimes see speed changes in the result. Why is that?

Thanks for the reply, I wrote with "Google translate" :) Vilmantas Michnevičius Lithuania vili@meganet.lt

[time4tea]

The speed calculations are simply using dist/time between points that are approx 3 seconds apart. It is possible that there is a bug when using videos and GPX... If you are able to share your video, the GPX file, and the command line you are using I could take a closer look. Please don't feel obliged to. If you can, please use WeTransfer or similar and send to gopro-overlay [at] time4tea.net.

[time4tea]

There are a few new controls in the latest version 0.78.0 - it might make it "good enough" for tunnels? maybe.

[vilmic1]

Thank you. The problem persisted in version 0.78.0. I will write to you at gopro-overlay [at] time4tea.net. when I prepare the files, I will describe the situation and find a place to put the information (the files are large). I will describe other issues I can demonstrate with these files later. But first, let's deal with the speed issue. Vilmantas

[time4tea]

I'll put some answers to things that I think are generic and useful here - perhaps information overload!

• GoPro supplies the speed as part of the "metadata" track, along with the GPS co-ordinates. We don't know how they calculate it. They supply a 2D and 3D speed - we currently use the 3D speed. Perhaps this is a wrong choice in some cases. When a GPX file is used, we will use the garmin GPX speed extension, if present. If a FIT file is used, we will use the speed attribute, if present.
• The speed calculation in gopro-dashboard-overlay currently uses the distance between two points approx 3 seconds apart, calculates geodesic difference (not haversine) and then uses speed=dist/time to calculate the speed. This isn't smoothed or averaged. There are a few smoothing algorithms implemented in the code already for other reasons - we could use those. When the GPS track is accurate - like from a Garmin device, these speeds are pretty reliable.
• The timing and alignment of the "metadata" with the movie should be very accurate - but there are multiple ways that this is done, depending on the particular model of GoPro. Before Hero8 Black, it uses a less accurate method, from Hero8 Black, it can use the STMP field, which is based on the STMP of the first SHUT packet - the first video frame. There may well be bugs here, but for the files that I have tested, I get the same values as the gpmf-parser code from GoPro, for both methods of calculation.
• When using a GPX/FIT file along with a GoPro MP4 - this might be less accurate, as we use the GPS times to sync the data sources. I seem to remember that this can be out by a second or so.
• This software definitely does not do the right thing with concatenated files. It might work ok, but it probably doesn't. Its complicated and making FFMPEG work with lots of input files, and supporting all the various options for GPUs etc seems quite hard. The same will be true with 'cut' files - as we will be missing the correct metadata to do a proper frame sync.
• There may be some bugs I don't know about when merging GPX and GoPro files - I'll use your data and see. Thanks for the examples!

[time4tea]

• I notice in the GPX file you created, you're using a GPX 1.1 file, but using a <speed> element. This software doesn't read that... its because in GPX 1.1 there is no speed element... This might account for the confusing behaviour...
• There is no particular reason that we couldn't read a non-conforming file. I only implemented the Garmin Extensions.
• This speed will be parsed ok

  <trkpt lat ="47.37736003508461" lon ="8.539389877469207">
      <ele>444.746359774318</ele>
      <time>2022-05-26T17:34:49Z</time>
      <extensions>
        <gpxtpx:TrackPointExtension>
         <gpxtpx:speed>3.77</gpxtpx:speed>
        </gpxtpx:TrackPointExtension>
      </extensions>
      </trkpt>

this speed will be ignored

<trkpt lat="55.2359580" lon="10.4869030">
        <ele>78.8</ele>
        <time>2022-07-09T07:09:31Z</time>
        <speed>123</speed>
<trkpt>

its just that a standards-conforming generator could not create this element...

[time4tea]

It seems that gpy.py (correctly) won't parse speed element from GPX 1.1 file. So I probably won't add this capability :-)

[KyleGW]

Great Explanation above! This should go into the documentation somewhere for sure, perhaps a page for "Technical explanation of speed calculation and data synchronization". It gives great insight into how it all works.

I am currently using "cut" segments to ensure I get the journey map tracks I want for the overlays. I do this manually at the moment, I am working on a script that will do "dynamic zoom" where it will cut 30 second segments of a video, run the overlay program against each segment in progressively narrower or wider zoom levels, and then stitch those all together. I will report back on how well this works since you indicate that the use of this on 'cut' segments will be missing the data for a proper frame sync. I also want to be able to at the very beginning of a video, start way zoomed out (Zoom=1) then progressively zoom in, giving the viewer context for where the action in the video is taking place. I imagine that cutting the right number of frames and getting the zoom right for each segment to make it smoothly animate will take a bit of work.

Loving the updates and feedback, this project is awesome! Great work!

[time4tea]

I probably overstated it a bit saying "probably doesn't" - it may well be ok! - but it can't use the "accurate" method from a cut thing - but it should be still ok. Really interesting to hear about the dynamic zoom ideas! (i think this is a feature request too) - use cases are great to heat about - and algorithms too. Thanks!

[vilmic1]

I created the speed_...gpx files purely for testing purposes. Format `

78.8

      <time>2022-07-09T07:09:31Z</time>
      <speed>123</speed>

` I chose without looking at any standards (thought to put the speed line). I do not use this file in the overlay generation (there is a file whose name starts with an underscore _). I generated this file only so that I could open it in a text editor and see what speed at a certain moment I expect to get the result in the video file, i.e. it is for comparison purposes only and for finding and demonstrating errors, inconsistencies. _...gpx file that is fed to the gopro-dashboard.py program is a modified original gpx file obtained with the gopro-to-gpx.py program. The number of points and the time stamp of each point correspond to the data of the original gpx file, only the coordinates and altitude have been changed. This _...gpx file is created from the original gpx file with my app by eliminating incorrect sections and recalculating the coordinates and height at each point. It makes no difference to use a geodesic distance or a haversine distance of 3 seconds. But if I feed my gpx file to gopro-dashboard.py, I expect the speed to be calculated from this file (adjusted by me) instead of 2D or 3D taken from the GOPRO GPS track (which is definitely wrong at the time). All _...gpx files (adjusted by me) are provided in the examples.

[vilmic1]

I see that more people have appeared in the discussion, so I will explain the situation. I prepared sample files, gpx files and generated result overlay files and hosted them on my server (> 100 GB). Here is the content of the Readme.txt file that is there:

Directory "The original files" - files from GOPRO6. Directories "GH010268" and "GH020268" - processed separate files. Directory "GH020268-GH050268" - four files joined with gopro-join.py program -> GH000268.MP4 Directory "Full movie" - all files joined with gopro-join.py program -> GH100268.MP4 All processed, adjusted file names start with an underscore character _. Time zone 'Europe/Paris' used during processing. Date and time in the widget +2 hours relative to the timestamp in the GPX file. The progress of the processing process is recorded in the log file. The goprol-1920x1080.xml graphics layout file was used to create the overlay.

speed_...gpx file created for testing purposes. The speed in km/h is recorded at each point. Speed is calculated between the current point and the point three seconds ago using the Haversine formula without estimating the change in altitude.

...gpx file that is fed to the gopro-dashboard.py program is a modified original gpx file obtained with the gopro-to-gpx.py program. The number of points and the time stamp of each point correspond to the data of the original gpx file, only the coordinates and altitude have been changed. This ...gpx file is created from the original gpx file with my app by eliminating incorrect sections and recalculating the coordinates and height at each point. After the track correction steps that I wrote about on github, I still perform (my app) integration (averaging) at each point (separately for each coordinate and altitude). For averaging, I came up with the following algorithm: I take 30 points forward and 30 points (configurable 60/2) back from the integrating point. At the beginning and end, I take as many points as are left. I am removing 60/3 Minimums and Maximums from this list. 60/3 points remain, from which I calculate the average. I record the average at the point to be integrated and go to the next point. A sliding window is obtained where the backward 30 points are already adjusted and the forward 30 points are new, i.e. the corrected values participate in the new integration. I chose 60 points for integration in the way of experiments (I set it as a parameter) so as not to blur the turns and speed changes too much.

Problems: 1) the speed in the widget does not always correspond to my expected value. It can be checked according to the date and time widget displays in the video and the speed value in the speed_...gpx file at the relevant time. This is clearly visible in the directories "GH020268-GH050268" and "Full movie" starting at time 7:29:30 in the widget (2022-07-13T05:29:30.000000Z in the gpx file). Sometimes the speed increases a lot, sometimes it becomes zero, although it should not be according to the gpx file. It seems that the speed behaves differently in the movies in the "GH020268-GH050268" and "Full movie" directories, although in theory it should be the same or similar. You need to compare both videos from 7:29:30 to the end. 2) The position on the map does not correspond to the filmed image. In the directory "GH020268-GH050268" at the beginning of the movie I pass under a bridge (overpass). The map already shows going under the railway bridge (6:54:46.7), but the camera shows going under the bridge later (6:54:49.0). Towards the end of the film, the discrepancy diminishes and disappears at the end. The longer the film, the greater the discrepancy at the beginning. In some films it is smaller, in some it is larger. Almost all films over an hour long have a discrepancy. The discrepancy in the "Full movie" is even greater, so I split this movie into three parts to get an acceptable result. I can partially solve this by recalculating all the timestamps of the gpx file a few seconds forward (backward), the image aligns with the map at the beginning (not always successful), but there are still problems with the speed (the speed does not match the image). However, if I take one file (not using join), then everything is fine (at least in this case). Examples are in the directories "GH010268" and "GH020268". There is no such problem here, although I have had cases where there were discrepancies in one file as well. 3) the speed does not match the image. The car is stationary, but the speed widget shows that it is moving. This is especially well visible in the "Full movie". At the beginning of the movie, the car is stationary, but the speed increases and the map "runs" forward. If I take one file (without join) everything is fine. Example "GH010268". GH020268-GH050268 also has the same problem. At tunnels, the car stops and the speed is still displayed, or vice versa. Maybe this problem and 2) are related, but since the speed has another problem (inconsistency in gpx), it now seems that 2) and 3) are separate problems.

I am not using gopro-cut.py in these experiments. If the files are trimmed (beginning or end), the situation changes a lot. Sometimes it gets better, sometimes it gets worse.

2) and 3) problems are probably not in your programs, but in the GOPRO files (I still have many GOPRO9 files, which are even worse than GOPRO6), maybe they lack data when combining files and ffmpeg does not know how to do it correctly. I tried joining the files using ffmpeg (not with gopro-join.py) but got the same result. I'm not experienced with ffmpeg, I tried it for the first time, and I have no knowledge about movie tracks, where timestamps are and how they are used, I'm just getting started. I see you have more experience, so you need to figure out how to work around these problems. Maybe if the data is missing, generate them fictitiously based on the nearby data, maybe the gps track needs to be shifted in relation to the video track (if they have separate time stamps), maybe the merging should not be done with ffmpeg, but in real time in your program, I don't know. So far I don't have the knowledge for that, I can only edit the gpx file. But without fully understanding how the whole system works, and adjusting the gpx, I can't always achieve the desired results.

True, since the GOPRO9 was hanging upside down by the windshield during the trip, it captured and recorded GPS worse than the GOPRO6 (at the side door of the car). Sometimes with ffmpeg I transfer the GPS track from GOPRO6 to GOPRO9 or vice versa (whichever track is better). I've already learned that much, but the time stamps sometimes don't match (and I haven't learned to match them yet), so the result is sometimes not good. Sometimes it is necessary to cut part of the GPS track from one camera and part from another camera, so it would be very nice to see a tool for such editing of GPS tracks in your program list in the future.

[time4tea]

Ah, OK. I hope I am getting to understand the core issue. I hope I can summarise it as:

Speed is incorrect when using gpx file and gopro movie together.

Using a gpx file, in addition to a gopro movie, one would expect the speeds to be calculated from the times and gps points from the gpx file, in preference to those in the movie. This doesn't seem to be the case.

Sorry if I'm a bit slow to catch on.

[time4tea]

I think there is a secondary issue about joining movies and how the timestamps work there. The current solution is pretty naive, so I expect there are some cases that it will work for, and others not. This might be a bit more challenging to fix...

[vilmic1]

Really so. I think (and this is correct) that the speed should be calculated from the coordinates in the gpx file, if there is no speed described directly in it. If there is a gpx file, the information in it must have priority over the information in the GOPRO file.

[vilmic1]

I think the join-cut issues should be dealt with later. Now you should deal with the speed.

[time4tea]

Really so. I think (and this is correct) that the speed should be calculated from the coordinates in the gpx file, if there is no speed described directly in it. If there is a gpx file, the information in it must have priority over the information in the GOPRO file.

Yes - this is how it is supposed to work! Lets see what is going on...

[time4tea]

ok - so first thought was that my speed calculation was totally wrong, so plotted the various speeds. Yours from "gps-with-speed" file, my calc based on your corrected GPX file, my calc direct from MP4, and GoPro supplied value.

Result is this:

So we can see a few things:

• Broadly, values are the same, once GPS has settled down.
• My calc is about 3 seconds in front of yours, and about 1.5 seconds in front of the GoPro supplied value.
• My calc from GPS points, is all over the place to start with - while GoPro value is variable, it is more stable. You have a flat value here - probably GPS was not reliable, so you edited the points.

So we can see that, while there is certainly a question of exact time alignment +- 3 s, and there is definitely some room for a smoothing applied to calculated speeds, the outputs are not crazy. The GoPro-calculated values are pretty OK.

Investigation continues...

[time4tea]

I think I know what is happening now. When the GPX file is merged with the GoPro data - all the GPS Points in the GoPro data are overwritten with the data from the GPX file. However, the GPX file doesn't contain any speed information. So this speed information "device speed" is retained. When the speed calculation is performed later, it puts in a new data item "calculated speed". When the "speed" is used later for the movie, "speed" actually looks first for "device speed", then for "calculated speed", because "device speed" will usually be more accurate. As the "device speed" was retained from the original points, you see this in the movie. There are a number of ways around this, including a code change, but to see the change immediately, you could try using metric="cspeed" in your layout file. This will force use of "calculated speed". This is just a theory right now, I'll attempt to prove it to myself.

[time4tea]

Here is a graph of the "device speed" and "calculated speed" after the GPX and gopro files are merged. I think it shows that the theory is correct?

[vilmic1]

I tried metric="cspeed" with a file in the directory "GH020268-GH050268". The result is the file c_GH000268.MP4. Everything is fine. Speed jumps are gone. The speed is the same or very similar to what I expect from a gpx file. I am satisfied with this decision. Only your page needs to describe how metric="speed" and metric="cspeed" work with a gpx file. Just so everyone knows that when using a gpx file without the specified speed, you need to use metric="cspeed" in the layout file. Then one more question. You say there is speed in the GOPRO GPS track. Why doesn't gopro-to-gpx.py output it to a gpx file? It would be possible to use different variants: with the calculation of the speed from the coordinates and with its indication directly in the gpx file (in the way that someone needs). Thanks for this solution, the synchronization and merge-cut problems remain (unless you try to somehow improve the speed calculation)

[time4tea]

Great! Yes - docs can be improved for sure. I think the current behaviour is confusing, and not really what people will expect, so I think I'll change it so it does the expected thing. You say "why doesn't it...." - but there is no why - it just doesn't. I didn't think about it, nobody asked for it. This would be the same answer for so many features! But it is a good idea! Thanks for your interest in the software and raising an issue!

[vilmic1]

As I already wrote, my knowledge of English is not enough to express my thoughts freely. I use Google translate. However, it also does not work perfectly (to put it politely). The rules for composing Lithuanian and English languages, words and sentences are fundamentally different. The Lithuanian language (together with Latvian) is the oldest and least changed living language of the Indo-European language group. The Lithuanian language is similar only to Latvian and is not similar to any other language in the world, but similar words can be found in languages that come from the same root and are archaic, little changed since birth. So Goggle translate sometimes doesn't handle the translation very well, and to convey the idea correctly I have to use constructions that are not used in the Lithuanian language. Sometimes I have to change the sentence structure several times to get what I'm thinking about. However, my knowledge of the English language does not allow me to accurately assess whether Goggle translate conveyed the idea correctly. So, don't be offended if somewhere my thoughts, suggestions seem like a demand or a reproach. Consider these to be translation issues and ignore them. Good luck with your work, your work is important to a lot of people, so let's not focus on the details.

[time4tea]

oh - My apologies - I didn't mean it in that way at all - I am not offended. I just mean, if it works in a dumb way, it may well be just working in a dumb way, not for a particular reason. It is great that people raise issues, so then I can see what people want to do, and make it work the right way for them. I really appreciate the effort you have gone into, explaining all this via Google Translate! Thank you for your kind words.

[vilmic1]

Hello. I tried version 0.80.0 with merged files in "GH020268-GH050268" and "Full movie" directories.

1. When using a gpx file, the problem of speed jumps is not present with metric="speed" . The speed fully corresponds to the information provided in the gpx file.
2. The problems 2) and 3) I wrote about earlier (mismatch of map and speed in the captured image in merged files) remained. However, I ran tests with the --use-gpx-only option: a) using gopro-dashboard.py without --gpx has a problem. Timelapse Factor = 0.994. b) using gopro-dashboard.py with the --gpx option and the gpx file I provided has the problem. Timelapse Factor = 0.994. c) Using gopro-dashboard.py with --gpx and --use-gpx-only options no longer has the problem. The map and speed correspond to the captured image. Can be checked by bridges and overpasses overhead. I put files that start with o (.mp4 and .log) on the server. However, in this case Timelapse Factor = 1.000 is visible in the log. Full command line: ~/venv/bin/gopro-dashboard.py --profile nvgpu --layout-xml ~/GOPRO/goprol-1920x1080.xml --use-gpx-only --overlay-size 1920x1080 --gpx $2 $1 $1. My conclusion is that gopro-dashboard.py is either doing something wrong with the synchronization between the video track and the GPS track, or the Timelapse Factor is being calculated incorrectly. I already noticed long ago that if the Timelapse Factor is not equal to 1, then the map and speed do not correspond to the recorded image. I just couldn't check it before. I thought it was a problem with the file join (gopro-join.py). But now I see that gopro-dashboard.py is handling data (video and GPS) and Timelapse Factor incorrectly. Just export the gpx data from the merged GOPRO mp4 file with gopro-to-gpx.py and then feed the same mp4 and gpx file to gopro-dashboard.py with the --gpx and --use-gpx-only options and the problem is gone . The map and speed correspond to the captured image.

[time4tea]

This is interesting! - Thanks for the info.

[time4tea]

although i didn't address everything in this thread, I think most cases are now working? there are certainly outliers but hopefully its mostly ok now?