Datum (was: 3D Position but no Altitude?)

[sumps]

We say in the spec that Position is in 3D but in all the examples I have seen it is just Lat/Lon....

"values": [ { "path": "navigation.position", "value": { "source": {}, "timestamp": "2016-02-17T14:1:19.000Z",
"longitude": 24.933841705322266, "latitude": 60.20331954956055

I assume to add altitude you just put "altitude": 123 and that the value is in meters but it would be good to confirm this and also confirm that the Altitude value stored is adjusted for Geoidal Separation or not.

[tkurki]

Schema at https://github.com/SignalK/specification/blob/master/schemas/definitions.json states altitude units as meters. Technically none of triplet properties are required, latlon probably should be.

Please annotate description and post PR?

[tkurki]

@timmathews where should we document the Geoidal Separation stuff in your opinion?

[sumps]

And even more importantly documenting the WGS84 datum issue as this has a much greater affect on safe navigation than the altitude being wrong.

[timmathews]

Have a look at this, and let me know what you all think: http://signalk.org/api/gnss-field-guide.html. That was something I started on very early on in the SK project before we had started using JSON-schema. I don't know why that stuff never made it in to the schema. I guess I ought to add it.

As for identifying the datum, I don't think it needs to be explicitly stated. If we know what GNSS satellites we're talking to, we know what reference frame is being used.

• GPS uses WGS84
• GLONASS uses PZ-90
• Beidou uses CGCS2000
• Galileo uses GTRF

However, if you tell me that there are GNSS receivers which automatically do the conversion internally, then my opinions are subject to change.

That said, if there are receivers which do this, how do they communicate the fact that they've done so? As far as I know there isn't anything in either NMEA spec which holds that information. At this point, we're probably talking about supporting manufacturer proprietary sentences or their binary interfaces instead. Not that I'm opposed to that, often you get better info from a GPS receiver if you can communicate with it via some protocol other than a NMEA one.

Where we absolutely do need to record the datum is in the section of the schema where we're storing routes and waypoints. In addition to the coordinates for each waypoint, we need to store the datum that those coordinates use. And there are way more than the four I referenced above. A stupid huge number in fact.

Converting between two datums to within a few meters isn't particularly difficult, a little bit of linear algebra and a lookup table are all you need. If more accuracy is required, then things get kinda tricky.

[faceless2]

I have to be honest, the datum differences was news to me. I had assumed WGS84 was universal - very interesting!

I realise one option is to store the datum for the coordinates in the model, or presume it is derived correctly from the GNSS source. But surely the better model is to convert all coordinates to the same datum - probably WGS84. The reasoning for this decision is the same as it would have been for converting all units to SI units - SignalK is supposed to hide precisely this sort of source-dependent data, and present a unified, standardised view in a source-agnostic manner.

It's more of a headache for the SignalK server, but I think it's fair to say it has to be done somewhere in the system - client or server. Doing it once, correctly, on the server means clients know they're getting the same coordinate system regardless of the source of data.

@timmathews said "If we know what GNSS satellites we're talking to, we know what reference frame is being used." To my mind that's precisely the situation we want to avoid. Example. I have a GLONASS and GPS reciever on my boat and one goes down, then either

1. the SignalK server has done the translation of all coordinates to WGS84. The client doesn't even notice the change.
2. the SignalK server is reporting raw coordinates, and the datum they're in. The client is monitoring the datum and the coordinates, doing the conversion itself, and everything continues to run smoothly.
3. the SignalK server is reporting raw coordinates and the datum. The client doesn't know or care about datums, and suddenly my boat jumps 20m to the north on the moving map.

I expect option 2 is the least likely of all these scenarios. To paraphrase, a man with one GPS knows exactly where he is. A man with two is never quite sure.

So, I've done a bit of research and it looks like converting from:

PZ90 to WGS84 (for GLONASS): can be done "with meter level accuracy". Formula at http://www.navipedia.net/index.php/Reference_Frames_in_GNSS, source paper at http://users.auth.gr/kvek/750-613_BOUCHER-itrs-pz-wgs84.pdf.

GTRF to WGS84: this might be even easier. From ftp://itrf.ensg.ign.fr/pub/itrf/WGS84.TXT:

New realizations of WGS84 based on GPS data, such as WGS84(G730, G873, G1150 and G1674). These new WGS84 realizations are coincident with ITRF at about 10-centimeter level. For these realizations there are no official transformation parameters. This means that one can consider that ITRF coordinates are also expressed in WGS84 at 10 cm level

I haven't actually seen any Galileo output, but I presume it comes as Lat/Lon/Height. What the above is saying is that when converted to ECEF, the coordinates will agree with WGS84 ECEF coordinates to within 10cm, which I think is enough. Converting from LLH to ECEF means you need the ellipsoid, and at http://itrf.ensg.ign.fr/faq.php?type=answer they recommend GRS80, which was what WGS84 was based on. So to me, it looks like Galileo coordinates and WGS84 coordinates can be considered identical, certainly to within 1m. For our purposes I think that's identical enough.

CGCS2000 - not looked into this yet.

This is all a bit beyond my pay-grade, so much of what I've added above might be wrong. However that's all the more argument for doing this on the server, rather then leaving this up to clients processing the data.

[faceless2]

163

[keesverruijt]

Couldn’t agree more, @faceless2!

On 07 Mar 2016, at 16:38, faceless2 notifications@github.com wrote:

I have to be honest, the datum differences was news to me. I had assumed WGS84 was universal - very interesting!

I realise one option is to store the datum for the coordinates in the model, or presume it is derived correctly from the GNSS source. But surely the better model is to convert all coordinates to the same datum - probably WGS84. The reasoning for this decision is the same as it would have been for converting all units to SI units - SignalK is supposed to hide precisely this sort of source-dependent data, and present a unified, standardised view in a source-agnostic manner.

It's more of a headache for the SignalK server, but I think it's fair to say it has to be done somewhere in the system - client or server. Doing it once, correctly, on the server means clients know they're getting the same coordinate system regardless of the source of data.

@timmathews https://github.com/timmathews said "If we know what GNSS satellites we're talking to, we know what reference frame is being used." To my mind that's precisely the situation we want to avoid. Example. I have a GLONASS and GPS reciever on my boat and one goes down, then either

the SignalK server has done the translation of all coordinates to WGS84. The client doesn't even notice the change.

the SignalK server is reporting raw coordinates, and the datum they're in. The client is monitoring the datum and the coordinates, doing the conversion itself, and everything continues to run smoothly.

the SignalK server is reporting raw coordinates and the datum. The client doesn't know or care about datums, and suddenly my boat jumps 20m to the north on the moving map.

I expect option 2 is the least likely of all these scenarios. To paraphrase, a man with one GPS knows exactly where he is. A man with two is never quite sure.

So, I've done a bit of research and it looks like converting from:

PZ90 to WGS84 (for GLONASS): can be done "with meter level accuracy". Formula at http://www.navipedia.net/index.php/Reference_Frames_in_GNSS http://www.navipedia.net/index.php/Reference_Frames_in_GNSS, source paper at http://users.auth.gr/kvek/750-613_BOUCHER-itrs-pz-wgs84.pdf http://users.auth.gr/kvek/750-613_BOUCHER-itrs-pz-wgs84.pdf.

GTRF to WGS84: this might be even easier. From ftp://itrf.ensg.ign.fr/pub/itrf/WGS84.TXT:

New realizations of WGS84 based on GPS data, such as WGS84(G730, G873, G1150 and G1674). These new WGS84 realizations are coincident with ITRF at about 10-centimeter level. For these realizations there are no official transformation parameters. This means that one can consider that ITRF coordinates are also expressed in WGS84 at 10 cm level

I haven't actually seen any Galileo output, but I presume it comes as Lat/Lon/Height. What the above is saying is that when converted to ECEF, the coordinates will agree with WGS84 ECEF coordinates to within 10cm, which I think is enough. Converting from LLH to ECEF means you need the ellipsoid, and at http://itrf.ensg.ign.fr/faq.php?type=answer http://itrf.ensg.ign.fr/faq.php?type=answer they recommend GRS80, which was what WGS84 was based on. So to me, it looks like Galileo coordinates and WGS84 coordinates can be considered identical, certainly to within 1m. For our purposes I think that's identical enough.

CGCS2000 - not looked into this yet.

This is all a bit beyond my pay-grade, so much of what I've added above might be wrong. However that's all the more argument for doing this on the server, rather then leaving this up to clients processing the data.

— Reply to this email directly or view it on GitHub https://github.com/SignalK/specification/issues/177#issuecomment-193303423.

[faceless2]

From wikipedia on GLONASS (https://en.wikipedia.org/wiki/GLONASS)

Since 31 December 2013, version PZ-90.11 is being broadcast, which is aligned to the International Terrestrial Reference System at epoch 2011.0 at the centimeter level

So that's three out of four all effectively using the same reference datum, or at least datums that are within a few cm of eachother. This is shaping up to be a bit of a non-issue, and that is certainly my favourite kind of issue!

[sumps]

Let's not lose sight of the original issue, which was clarifying the Altitude value in the 3D position and confirming how we handle horizontal and vertical datums. We need to make a decision on whether we simply state that all positions in Signal K are WGS84 and decide the best place to state this, so that there is absolutely no chance of this important information being missed.

The satellite status issue is being dealt with in #163

[keesverruijt]

Yup. What would be helpful is if the model would explain some datapoints a little better than existing standards, in plain english. Ellipsoid deviation -- fine, but what does it actually mean. Same goes for a lot of AIS and GNSS terminology, like DOP, RAIM, AIS communication state. Otherwise we'll all just zone out I'm afraid.

On a further note, I found that N2K has a PGN for "Local Datum", and lo and behold my Airmar PB200 even sends it out (with local datum set to W84, delta long/lat/alt set to zero, with W84 as the reference datum):

2011-04-25-10:16:39.293 6  36 255 129044 Datum:  57 38 34 FF 00 00 00 00 00 00 00 00 00 00 00 00 57 38 34 FF
2011-04-25-10:16:39.293 6  36 255 129044 Datum:   W  8  4  .  .  .  .  .  .  .  .  .  .  .  .  .  W  8  4  .
{"timestamp":"2011-04-25-10:16:39.293","prio":6,"src":36,"dst":255,"pgn":129044,"description":"Datum","fields":{"Local Datum":"W84","Delta Latitude": 0.0000000,"Delta Longitude": 0.0000000,"Delta Altitude":0.000000,"Reference Datum":"W84"}}

[keesverruijt]

So we can say that everything is in W84, and local desired datum can be sent out by the server. Like the way UNIX does time, not how Windows does it.

[sumps]

That gets my vote and also for the vertical datum we store the raw altitude value in meters and the geoidal separation value and then the server can send out the adjusted altitude if required.

[keesverruijt]

raw altitude, is that difference from the geoid or from mean sea water level? The latter seems more useful and I think it is what my N2K GPS produces, but I'm not sure whether that is what you are suggesting?

[sumps]

As a final note on this issue, we have now implemented Datum support in iKommunicate. All positions will still be converted and stored in the Schema as WGS84, but we will also include the values from the received Datum sentence or PGN.

// 20160408114445 // http://192.168.0.106/signalk/v1/api/vessels/self/navigation/datum

{
  "datum": {
    "deltaLatitude": {
      "value": 0
    },
    "deltaLongitude": {
      "value": 0
    },
    "deltaAltitude": {
      "value": 0
    },
    "reference": {
      "value": "W84"
    },
    "local": {
      "value": "W84"
    }
  }
}

I now need to raise a PR for this new Datum object.

[tkurki]

The deltaLatitude/Longitude/Altitude are based on the datum, but they are actually "meta"data about a particular position and should be optional siblings of lat/long/alt in the position in definitions.json. This way all positions gain these optional properties.

The datum itself under navigation.datum can be linked from a position via the source.

Datum Signal K delta messages are sent periodically, in practice as a gateway/server receives datum messages. DeltaLatitude/Longitude/Altitude updates are sent optionally with each delta Signal K message as part of the position object:

{
  "updates": [{
    "source": {
      "label": "n2kFromFile",
      "type": "NMEA2000",
      "pgn": 129038,
      "src": "43"
    },
    "timestamp": "2014-08-15T19:01:08.763",
    "values": [{
      "path": "navigation.speedOverGround",
      "value": 6.99
    }, {
      "path": "navigation.courseOverGroundTrue",
      "value": 4.4838
    }, {
      "path": "navigation.position",
      "value": {
        "longitude": 24.919905,
        "latitude": 59.835495,
        "deltaLongitude": 0.00023,
        "deltaLatitude": 0.000011
      }
    }]
  }],
  "context": "vessels.urn:mrn:imo:mmsi:248071000"
}

Closing #208, as it does not work like this.

[tkurki]

I assume Reference + Delta = Local, but if everything in SK is WGS84 we can omit local.

If we remove deltas and local we are left with just single string, so it could be directly under navigation.datum.value. What namespace is used for the strings - what abbreviations do we use? And is this now datum or spatial reference system (SRS)?

https://en.wikipedia.org/wiki/Spatial_reference_system

Spatial reference systems are defined by the OGC's Simple feature access using well-known text

https://en.wikipedia.org/wiki/Geodetic_datum

[sumps]

OK - this seems to be the last issue that needs sorting before V1 release.

I still stand by my original feeling that we need to have a datum object that stores the datum information sent by a positioning system in NMEA0183 or NMEA2000. We have implemented this in iKommunicate as detailed above and it would be good if we could just make this the Signal K implementation.

We still have this issue with the way Signal K stores Positions, as has recently been raised by a few people where we need to restructure the key so that Latitude, Longitude and Altitude have a value field.

[joabakk]

Ref #171, we might consider adding a z position of the antenna location onboard as well, in addition to A,B,C and D

[sumps]

That makes a lot of sense although how many marine users will make use of altitude is questionable. I guess someone sailing on a mountain lake or a canal/river boat that has just finished a flight of locks might have some interest.

[joabakk]

Granted. But I don't think we should judge what properties should be useful. It could be used e.g. for multiple GPS inputs, much like how DP systems are set up.

• If you have three, you could weight the inputs more on the two closest positions.
• If you only have two, you could compare the altitude to the known altitude of the antenna to know which one to trust more. The more information available, the better (almost)

[sumps]

Sorry re-reading my last comment, I came across as a bit negative, this was not my intent and I think GPS antenna height should be added.

[rob42]

FWIW the original thought behind altitude was more about wave motion - that a sensitive IMU/altimiter might be able to provide data for deriving wave motion, and that might benefit autopilot/weather etc...

[sumps]

This type of sensor would not require an antenna height value as it would be measuring changes rather than absolute altitude.

[joabakk]

Datum makes for fascinating reading, see http://geospatial-solutions.com/nightmare-on-gis-street-accuracy-datums-and-geospatial-data/ Apparently there is no exact SI-type definition. WGS84 uses the position of the north pole from 1984 as a reference... Nevertheless, I think we should add datum as metadata to the position source, and we are ready for what may come in the future.

[sumps]

The problem is that the Datum information comes in at a different time and in a different sentence/PGN than the position and the gateway cannot handle the pairing up of the position and datum information required to add datum as metadata to a position.

[joabakk]

If this issue is standing in the way of releasing V1, then I would argue that WGS84 is currently the de facto standard and can be applied as default. If and when units come that do not translate to WGS84, the user or implementation would have to configure the other datum. This is an implementation issue, not a specification issue