Mapping GGA Quality to NavSatStatus can be misleading

[ablakey]

The quality values used to but generally no-longer reflect position solution source but rather solution quality. Mapping quality to NavSatStatus.status seems to be misleading and removes meaningful information.

For example, mapping 4 and 5 to GBAS is problematic because:

• 4 is almost exclusively fixed-integer RTK (1-2cm)
• 5 is commonly float RTK (10-100cm as it converges to fixed)
• 5 is also commonly satellite-based corrections like OmniStar, TerraStar (10-20cm)

Therefore use of NavSatStatus.status can significantly mislead when we have a quality solution. For example, I couldn't rely on this value to know when to capture a measurement as a GPS datum for a world to local transform (do I have 1cm, or 100cm accuracy?)

Is there a possible resolution that uses existing messages? Do we need a new message? This might warrant a greater discussion on the importance and strategy of preserving and communicating quality data in greater detail.

[mikepurvis]

What's the relationship between GGA quality and hdop/pdop/vdop values? Isn't that where you're supposed to go for a more precise sense of how accurate the fix is?

[ablakey]

Yes, *dop values are ideally where you go to derive position accuracy if you want a measure. But the correction integer is commonly used as an indicator of when you reach certain quality thresholds.

If we're not trying to supply NavSatStatus.status as a convenient way to identify what quality ballpark NavSatFix positional data is in, what's the desired purpose of it?

[PaulBouchier]

I recently released a package that supported the swiftnav RTK GPS receiver into ROS and documented it here: http://wiki.ros.org/swiftnav_piksi on the wiki. I was subsequently approached by another company with a request to evaluate their RTK GPS receiver, which emits NMEA messages, including some non-NMEA-standard messages that describe the RTK mode and accuracy. Their product data is here: http://navspark.mybigcommerce.com/ns-hp-rtk-capable-gps-receiver/ and I think it's potentially interesting to roboticists because of its low price ($50 - $150 per receiver) and high accuracy.

The fundamental problem is that the ROS sensor_msgs/NavSatFix message doesn't have enumerations for RTK mode, and nor does NMEA standardize them. @ablakey asserts above that 4 is typically Fixed RTK and 5 is Float RTK. If this is really typical, can we change the definition?

A secondary problem is that the nmea-navsat-driver generates a covariance matrix that contains covariance values that never go below 1.0 (AFICT). This even though the receiver is reporting a fix with 4cm or so RMS error. Probably the driver doesn't know how to get the accuracy, or maybe there's a low limit - I didn't look at the code to see why that's happening.

With now two low-cost products that provide centimeter accuracy available to roboticists, I wonder if we should do something to address the problem. I can imagine adding enums to the NavSatFix message, and/or putting a product-specific fix into nmea-navsat-driver to understand particular products. Or maybe there are better approaches, such as parameterization.

Thoughts?

[PaulBouchier]

Make that three low-cost RTK systems. Ublox just announced. http://goo.gl/TjpBc2. IMHO ROS needs to get with the program & support RTK explicity.

[ablakey]

Hi Paul:

Short of a product survey, I can only confirm that my assertion about Quality integers is true for every GNSS I've used, which is 4 or 5 different brands (Novatel, Topconn, Trimble, Piksi, and likely some others).

I can't speak to your comments on covariance matricies. I haven't really used them for anything meaningful in my work.

I get a sense that the challenge is changing a well-established message. Breaking everyone's code is a non-option. Is there a reasonable solution that is backwards compatible, or would we need to author a new message? If so, that means we're adding a bit more noise to the signal: is it worth it? In my opinion, yes. As I stated above, I feel that the current implementation can be deceptive and therefore more harmful than good. But I'm not married to this idea and hope this can be an item of discussion among interested parties.

FWIW, I'm happy to support any decision made with software development effort.

[PaulBouchier]

Andrew - Thanks for the data point supporting the common use of quality integers.

Backing up your assertion of the meaning of status 4 & 5, this site: http://www.gpsinformation.org/dale/nmea.htm#GGA defines: 1 Fix quality: 0 = invalid 1 = GPS fix (SPS) 2 = DGPS fix 3 = PPS fix 4 = Real Time Kinematic 5 = Float RTK 6 = estimated (dead reckoning) (2.3 feature) 7 = Manual input mode 8 = Simulation mode I don't know how authoritative that site is, but if good, we could propose that this full enumeration be placed in the ROS NavSatStatus message. We could change nmea_navsat_driver to pass the fix quality straight through instead of remapping 4, 5 as it does today, subject to a FixQualityRemapping parameter that could default to remapping in Indigo/Jade and default to no-remapping in later releases. The remapping loses information, and knowing whether you're in RTK Fix mode is IMHO crucial because you can have a lot more confidence in the fix accuracy if so. It is also important to help sequence machine startup in a way that HDOP isn't: the receiver goes from state DGPS to Float RTK to Fix RTK. It may hang around in Float RTK for a while but you likely don't want to start the machine until it reaches Fix RTK because how do you know you can count on the accuracy when in Float RTK mode.

My reading indicates that HDOP is NOT an indication of fix accuracy - it is a multiplier to apply to whatever the fix accuracy is reported as by the GPS. It's lowest value is 1.0. It is useless for telling whether a GPS is in RTK mode with 3cm accuracy, or in RTK Float mode with some meters of accuracy. The NavSatFix message defines the covariance values in m^2 and you can't get from HDOP to m^2, so I think we have a bug in driver.py line 109, and a more general problem that apparently GPS receivers vary in how they report accuracy. (I see I have this bug in the swiftnav-piksi driver too.) Apparently Garmin reports accuracy in a proprietary message, swiftnav doesn't report it so receiver mode (e.g. Fix-RTK) can be used as a rough proxy, and SkyTraq uses a proprietary message. My experience in this domain is limited. Do you guys have any thoughts on how to deal with the variety of receivers with proprietary accuracy-reporting schemes? Maybe start out by decoding some of the various proprietary messages?

Paul

[ericperko]

Hi @PaulBouchier @ablakey, thanks for the info. I don't have much time to work on ROS stuff during the week anymore (so I haven't gotten a chance to read all of these comments), but I'll make some time this weekend to review things so we can start to discuss how to resolve these issues.

In the meantime, you may want to take a look at http://wiki.ros.org/sensor_msgs/Reviews/2010-05-31%20GPS%20Proposal_API_Review , which is the original review that got NavSatFix into sensor_msgs in the first place. Not sure how much of that reasoning applies to this discussion, but it may be relevant background info if you hadn't seen it before.

[ericperko]

So I think there are two issues in this thread that can be addressed separately.

1. How to represent the accuracy of a given fix message?
2. How does the driver map NMEA messages into that accuracy representation?

On issue 2, the driver is very basic right now and I have no problems with any improvements in this area. The current mapping of DOP->covariance is something hacky I made up for the first version of the driver. If there are additional messages that add more detailed accuracy information, any patches to support those would be appreciated. I never liked the idea of needing proprietary messages, but, as far as I know, the NMEA0183 standard doesn't actually have a message with anything better than DOP numbers. I've been unhappy with just using the data from NMEA0183 messages for autonomous operations - in basically every system I've worked with that had a GPS, we ended up reading the manufacturer binary messages so that we could get actual accuracy information direct from the INS.

On issue 1, I'm not yet convinced that we need to add more solution quality enumerations to the NavSatStatus message. If we could actually fill in covariance properly, is there a situation where the existing status fields + covariance are insufficient? In my experience, RTK mode by itself is pretty useless without the associated uncertainty info, which, with a better driver, would be represented in the covariance info (most of the INS systems (Novatel, Applanix) I've worked with have a message that includes at least the info to set the diagonal). I've never found a system where I trusted just the RTK mode to give me accuracy information - I've always used the reported uncertainty for whether to use a given position estimate. That uncertainty + whether the receiver was using corrections was sufficient to know whether to operate autonomously.

The existing fix type in NavSatStatus was mainly to help put an expected range on the covariance for consumers. For example, even if the reported covariance gets very small (<10cm) because a non-RTK receiver has been stationary for days and has averaged out a great solution, by checking the status field and not finding GBAS, a consumer (like an autonomous ground vehicle) can understand that the solution is unlikely to stay at such a high precision.

[PaulBouchier]

Thanks for weighing in on this Eric. Bottom-line, like you, I do not yet see a compelling case for changing the NavSatFix message definition, and I agree we should fix the accuracy-reporting issues in the driver. The message issues and the driver issues are separable and I agree with fixing the driver independently of consideration of message-level issues. I will create a new issue for discussion of driver fixes to report measurement uncertainty.

I will develop some driver fixes, and also add diagnostics such as I did for the swiftnav RTK gps driver, which you can see in section 5.1.2 here: http://wiki.ros.org/swiftnav_piksi

Some additional thoughts on the question of the NavSatFix message and its relationship to RTK GPS position below. This is not to advocate for changes, but to state where I think we are at.

Problem statement: 3 vendors are selling or soon will sell RTK GPS devices with best-case 2cm accuracy. These likely have some application to outdoor robotics. Vendors are asking for ROS support, to increase their market share. However there are some conceptual differences between inexpensive RTK GPS units and regular GPS or DGPS receivers. They may warrant some thought about the NavSatFix message around issues it has expressing the RTK GPS fix.

Review of 2010-05-31 GPS Proposal_API_Review

• Much careful consideration went into architecting the NavSatFix message. There is little that can be criticized. The fact it has lasted 6 years without change is testament to the quality of the work. However, technology changes, and it has caught up with the NavSatFix message.
• There was almost no consideration of RTK GPS. Architects could not have forseen emergence of inexpensive RTK GPS - it was appropriate then to pay little attention to it, but now we perhaps should consider what RTK GPS means and whether any changes are indicated. I suggest if we do so, we move the discussion over into a new issue in the ros/common_msgs repo.

I suggest we close this issue.

RTK GPS NavSatFix message Issues

• All other GPS techniques try to provide an absolute fix. Even differential (e.g. SBAS or WAAS) techniques involve a baseline to a surveyed location. The inexpensive GPS receivers can't assume that, since the user may have installed the base station, so the measurement provided is usually accurate but not absolute. This seems like an important distinction for system design - perhaps navigation SW should be provided this information. The location of the reference station may be self-surveyed with unaided GPS, or may be completely arbitrary, entered by a user. Only the implementer of a particular RTK GPS installation can know. If it is important for clients to know, then the message would be an appropriate place to surface that. Without changes, the location in the NavSatFix message may be absolute, or may be arbitrary.
• Non-differential GPS produces a scattered estimate of position from the outset. RTK GPS goes through phases, from a scattered estimate, to a moving estimate (Float RTK), to a fixed high-accuracy estimate. The phase transition to Fixed RTK is potentially important, for example as a gate that enables machine operation. Should it be reported to navigation or machine control SW for direct use? Or should navigation/other SW use variance as a proxy for the transition to Fixed RTK mode? I don't see the compelling argument for surfacing Fixed RTK mode transition separately from the error estimate, except for the diagnostic purposes alluded to in earlier responses. I propose adding gps mode to diagnostics, to help the user understand how the system is working.

Paul

[ablakey]

Academically I do believe the current format is possibly misleading. However, after the discussion, I agree that there isn't a compelling practical reason to change it now. +1 on closing.

[PaulBouchier]

One more thought, since we have the API review discussion link here:

One of the motivations for the nmea_navsat_driver seemed to be to eliminate the need for gpsd. Presumably gpsd, being an independent project, was considered an unnecessary/undesirable dependency. However, I came across this horror story http://esr.ibiblio.org/?p=801 by Eric Raymond, the author of gpsd. It explains the many layers of issues in NMEA and typical receiver implementations of NMEA.

The nmea-navsat-driver is so simple that it probably has some of the weaknesses that have been fixed in gpsd, for example, handling receiver-specific messages, some of which involve binary messages. It may not handle as broad a range of receivers as gpsd. OTOH, it's been around for a few years & presumably is good enough for roboticists.

Just wanted to record my learnings before we close this thread. +1 for closing.

[PaulBouchier]

I think Andrew or Eric have to close the issue - I can't. Can one of you guys please do so?