tumic0 / GPXSee

GPS log file viewer and analyzer with support for GPX, TCX, KML, FIT, IGC, NMEA, SLF, SML, LOC, GPI, GeoJSON and OziExplorer files.
https://www.gpxsee.org
GNU General Public License v3.0
937 stars 129 forks source link

Eliminating GPS outliers doesn't always work. #525

Closed dumbenis closed 8 months ago

dumbenis commented 8 months ago

I disabled the "Eliminate GPS outliers" setting in preferences/options to see GPS outliers in GPX files. In the display of a GPX file, the GPS outliers were not visible. See example: left Avenue, right GPXSee.

07174502
tumic0 commented 8 months ago

Please provide the GPS log file, without the data it is not possible to investigate the issue any further.

dumbenis commented 8 months ago

Uploading a GPX-file is not allowed. Help needed.

sikmir commented 8 months ago

Uploading a GPX-file is not allowed. Help needed.

Just put it into zip archive.

dumbenis commented 8 months ago

07174502.GPX.zip

tumic0 commented 8 months ago

I have analyzed the file and the main problem is the data is absolute crap. The whole file is full of different GPS positions with the same timestamp which is nonsense by definition. The outlier with the missing first digit (51N -> 1N) is just the icing on the cake.

You do not see the "path to Africa" because the pause detection, not the outlier detection algorithm, eliminates it. All the algorithms in GPXSee are designed to work on real data that may contain "real errors", but systematic errors like "some Chinese crap producer not writing sub-second timestamps when sub-second sampling is used" will break stuff. And that's ok as any attempt to handle even such crap data would almost certainly be at the expense of the "normal usage".

dumbenis commented 8 months ago

First of all thank you for your reply. The GPX data is part of a test of the Columbus P-10 Pro Submeter (0.5m) GPS/GNSS Data Logger and USB Receiver with Broadcom BCM47755 GNSS Chip. I know that with a built-in antenna sub-meter accuracy cannot be achieved. As an user, I am extensively testing this GPS logger because I experience various problems: the well-known and well-documented problem of outliers, invalid timestamps such as and unable to recover after signal loss. Columbus replaced my unit but I still experience the same problems. I investigate the test results and report errors to Columbus (Unknown Victory Co., Ltd. with Hotmail e-mail addresses) and their reseller in Germany, because the Columbus P-10 Pro is not a reliable GPS-logger now. In my error reports I use the Avenue and GPXSee applications. I saw the differences in graphic display and submitted a report to you.

You pointed out the problem of systematic errors in the GPX data of "some Chinese crap producer not writing sub-second timestamps when sub-second sampling is used". In the GPX data, I saw that a GPS coordinate is recorded five times every second. The Columbus P-10 Pro tested appears to use the 5Hz log mode, where it records 5 trackpoints per second. Your conclusion is correct: the Columbus P-10 Pro uses sub-second sampling without sub-second timestamps. So Columbus implemented sub-second sampling incorrectly.

Columbus highlights the 5Hz log mode in its advertising, but says nothing about the faulty implementation and another drawback: reduced accuracy & increase drift by low velocity scenarios(<20km/h). See gpswebshop.com. In short: I put the Columbus P-10 Pro in 1Hz log mode and continue testing.

Question: which reliable GPS logger would you recommend?

tumic0 commented 8 months ago

which reliable GPS logger would you recommend?

I have no idea. My personal GPS usage is limited to logging using the mobile phone. For over a decade I have been using a Nokia 2710 with Trekbuddy, now I have a Samsung S22 with GPS Logger. The S22 has a much better "signal", but the old Nokia was far better in terms of weight/size and battery life. My logs are MTB rides/cross country skiing somewhere between 20-90km so I really do not care about some sub-meter precision, I even use a distance filter of 10m when logging the data.

As GPXSee author I have seen various logs from different devices/services and even the "big companies" are able to produce crap. GARMIN is famous for their poor SW/FW quality and even Strava has been producing invalid GPX exports for a long time (maybe they still are). But note that my world-view is very skewed as most data I get in the bug reports is broken and I very rare get some of the (hopefully) majority data out there that is ok...