Calculate speed for tracks without <speed>

[njohnston]

Many GPX tracks don't have the <speed> tag: only position (<trkpt> or <rtept>) and an associated timestamp. For these tracks, OsmAnd shows no speed data:

As this track has a timestamp associated with each position/point, it is possible to calculate the speed, albeit slightly approximately depending on the geoid used.

Please calculate speed for such tracks.

[sonora]

GPS data normally contains a speed label for each point recorded, it is part of the GPS algorithm. May I suggest you also look into "Cardio Trainer", or "CardioTrainer - converted by CardioTrainerData 0.01" respectively, and check why there is no speed data included (did you perhaps select to drop it in the export?). Or perhaps open a ticket with them requesting to be able to keep it in the export?

Obviously, deriving speed from distance/time between 2 different measurements will result in another value / averaging than "speed as reported by GPS per measurement", even if the practical different may be insignificant in many cases ... I would hate to mix both in OsmAnd without ways to tell them apart.

[njohnston]

@sonora: CardioTrainer is abandoned--I still use it as it's a good workout tracking app. I wrote code to convert workout data (stored in protocol buffers) to GPX. There is no speed (per point) in the workout data.

My Tracks, another GPS logging app, does not include speed in its GPX files either. It looks like RunKeeper doesn't either.

Speed is not mentioned anywhere in the GPX specification (I realise this isn't the same as GPS itself, but it is designed to contain GPS data)--indeed, OsmAnd includes speed in <extensions>. Someone on the forum reports that other software even refuses to process GPX files generated by OsmAnd because of <speed>.

The more I think about it, the only reason I can think of for including speed in a GPX file is to improve user experience by not having to spend time calculating the speed every time the track is opened.

The speed that OsmAnd currently includes in tracks is from location.getSpeed(). It's unclear to me what time period and distance this is calculated from. Deriving the speed from the distance and time between the points is at least clearer: you know exactly how the speed is being calculated.

Is your only concern not being able to tell the different speed measurements apart? I'm not sure how to indicate that in the UI. From looking at the code (GPXUtilities.java) it shouldn't be too difficult to calculate the speed.

[sonora]

Yes, it is not difficult at all. My "gut feeling concern" is this: GPS readings (respectively what is derived from them) do contain "(GPS)speed" information, i.e. GPS readings have some sort of "built in motion detection". (Most common GPS chipsets I have tested derive and report report this. Also Android's getSpeed() method taps into this by reporting meters/second, and 0f only if not available.)

On the other hand, "(calculated)speed" as derived from secondary data via "perceived displacement over time" is a different animal. I believe (but would have to do more research) that it can e.g. happen that you record your position at rest over extended periods, and while "(GPS)speed" will always or mostly correctly deduce that you are at rest, "(calculated) speed" values would result in near random "speed" artifacts because of fluctuations in your deduced position. I am not saying that this is bad per se, but I am not sure off hand which value is generally more precise (or useless) in what situation, and I hate the fact that in OsmAnd we would report a mix of both without discrimination.

It may be a mute point, and I agree that for a "well behaved" track without speed data reporting a "calculated speed" is likely a useful feature, but I wanted to caution us from rushing into an implementation without thinking first :)

[sonora]

I guess from my point of view we could do it if this is really a main stream request. But I recommend we draw curves depicting "calculated" speed in another color (like red instead of orange) so we can distinguish that case from GPS reported speed.

[njohnston]

I have many GPX files without speed data--in fact, none of my GPX files except those generated by OsmAnd have speed data. There is no standard way of including speed information in GPX files, so even if speed was included, OsmAnd might not be able to read it. Therefore I think it's extremely important to calculate the speed.

Out of interest, why do you think the calculated speed is derived from "secondary data"? How is getSpeed() (or the underlying implementation) actually calculating the speed if not by measuring distance over time?

Using GPS Status on my phone, the speed value fluctuates significantly (between about 4 km/h to 10 km/h) when I'm walking at a consistent pace. This is with weak filtering enabled. Plotting such speed at high frequencies would lead to very "noisy" data and it's debatable whether this would even be more meaningful or accurate.

[sonora]

@njohnston Yes, it is a complex topic, and my terminology was by far not precise enough to honor this ...

In a nutshell, without exceeding what we can do in a blog like this, it is as follows: What I called "GPS speed" is a result of the GPS algorithm analyzing signals received from different satellites, you can picture it a little like phase shifts observed in these signals. This is pretty much an integrated part of position determination via GPS, and each such speed value is associated with one position estimate exactly. What you suggest, on the other hand, is deriving a speed value by taking the difference between 2 measured locations, and dividing by the time which had passed between the 2 measurements, which yields the average velocity needed to get from A to B in time, and in a straight line.

So if you are at rest at Point A, record a location fix, then walk to point B, stop and wait for some time, then take a second location fix, "GPS-speed" would usually report that you were at rest at both points, while "calculated speed" would report the "average" speed as if you were in constant motion. Plus, the latter value would inherit the errors from both position estimates.

Or a different example: Walk the same distance, in a straight line, at the same speed, twice in a row. On the first occasion, record a point once every minute. On the second occasion, record a point every second. Compare the "calculated length" of both tracks. You will find that your second recording appears a lot longer than your first (it would not surprise me if the difference is up to 30%). This is due to the bigger "zig zag" artifact the many repeated measurements may suggest you have walked. (But in fact you have not, do it sober! :) ). Your "calculated speed" will also reflect this difference, while GPS speed should be less sensitive to that.

Still, I am not saying that one is necessarily inferior to the other, neither have I thought about which one bears more "accuracy" (although I suspect it is "GPS speed"). For many or most everyday applications it may actually not matter much at all. And it is clear that both values will fluctuate, and for different reasons: The first already because when you walk or jog, your device will actually undergo an undulating motion depending where on the body you carry the device (upper arm, back pocket, etc.), the second by all artifacts (inaccuracy) GPS position measurement is subject to.

Filtering is, of course, an altogether separate topic. Ok?

[njohnston]

@sonora: thanks for the detailed reply. To summarise this issue so far:

• Both methods of calculating speed are valid
• OsmAnd should calculate speed where needed/possible
• OsmAnd should indicate the source of the speed

Agreed?

[sonora]

Yes, fully agreed from my side. :)

I often take the scientific perspective in our project, and it sometimes means to play the "devil's advocate", but does not necessarily meam all my comments are equally important to the casual user under all circumstances. The name of the game is sometimes to automate and simplify the UI and at the same time maximize functionality while keeping things "clean". Not always an easy task. ;)

[njohnston]

@sonora: I've looked at the code and made some changes. Please take a look and see if it's reasonable. I haven't implemented any indication of the method of calculation yet. I think we need some input from @vshcherb on that.

The change wasn't quite as simple as intended because OsmAnd sometimes doesn't insert <speed> for some points, so we cannot rely on the absence of <speed> to decide when to calculate the speed.

On my test track (a ~12 km walk), OsmAnd's current speed measurement reports an average speed of 7 km/h and a maximum of 10.5 km/h. I removed <speed> from that track and tested it with my code. The average speed is 6.9 km/h, and the maximum is 11.2 km/h. (I think both maximum values are due to false/inaccurate readings, so I'm not concerned by the difference.) The OsmAnd current measurement seems to more accurately report time moving.

diff --git a/OsmAnd/src/net/osmand/plus/GPXUtilities.java b/OsmAnd/src/net/osmand/plus/GPXUtilities.java
index 899e6ab..d5e7fef 100644
--- a/OsmAnd/src/net/osmand/plus/GPXUtilities.java
+++ b/OsmAnd/src/net/osmand/plus/GPXUtilities.java
@@ -312,6 +312,7 @@ public class GPXUtilities {

    public boolean hasElevationData;
    public boolean hasSpeedData;
+   public boolean hasSpeedInTrack;

    public boolean isSpeedSpecified() {
      return avgSpeed > 0;
@@ -394,15 +395,9 @@ public class GPXUtilities {
          }

          float speed = (float) point.speed;
-         Speed speed1 = new Speed();
-         if (speed > 0) {
-           totalSpeedSum += speed;
-           maxSpeed = Math.max(speed, maxSpeed);
-           speedCount++;

-           speed1.speed = speed;
-         } else {
-           speed1.speed = 0;
+         if (speed > 0) {
+           hasSpeedInTrack = true;
          }

          // Trend channel approach for elevation gain/loss, Hardy 2015-09-22
@@ -478,6 +473,10 @@ public class GPXUtilities {
            point.distance = segmentDistance;
            timeDiff = (int)((point.time - prev.time) / 1000);

+           if (!hasSpeedInTrack && speed == 0 && timeDiff > 0) {
+             speed = calculations[0] / timeDiff;
+           }
+
            // Motion detection:
            //   speed > 0  uses GPS chipset's motion detection
            //   calculations[0] > minDisplacment * time  is heuristic needed because tracks may be filtered at recording time, so points at rest may not be present in file at all
@@ -499,6 +498,15 @@ public class GPXUtilities {
          if (!hasElevationData && !Float.isNaN(elevation1.elevation) && totalDistance > 0) {
            hasElevationData = true;
          }
+
+         Speed speed1 = new Speed();
+         if (speed > 0) {
+           totalSpeedSum += speed;
+           maxSpeed = Math.max(speed, maxSpeed);
+           speedCount++;
+         }
+
+         speed1.speed = speed;
          speed1.time = timeDiff;
          speed1.distance = elevation1.distance;
          speedData.add(speed1);

My change includes a trivial refactor of this code too:

          if (speed > 0) {
            totalSpeedSum += speed;
            maxSpeed = Math.max(speed, maxSpeed);
            speedCount++;

            speed1.speed = speed;
          } else {
            speed1.speed = 0;
          }

Since speed1.speed is always assigned to, it can be moved out of the main body of "if", and the "else" can be removed, because if speed is 0 then assigning speed to speed1.speed will have the desired effect. No need to explicitly assign 0.

If you think my approach looks good I'll submit a pull request :)

[vshcherb]

Please submit pull-request.

[vshcherb]

Looks like the issue has been merged and fixed.