SignalK / specification

Signal K is a JSON-based format for storing and sharing marine data from different sources (e.g. nmea 0183, 2000, seatalk, etc)
Other
91 stars 68 forks source link

Suggestions on data format #521

Open ccmsi opened 5 years ago

ccmsi commented 5 years ago

Reading over the data format specifications some things came to mind which i would like to offer for your comments / consideration

Comments on Data Format & Data Management

Non breaking

Add covariance data definition for all values as optional but recommended where the information is available

Give sources optional a location relative to their owner (where are the sonar receivers 1 and 2 on the vessel) so their findings can be merged correctly.

Do not do automatic, implicit source preferencing as in http://signalk.org/specification/1.1.0/doc/data_model_multiple_values.html . Never make a value ambiguous. If there are two sources for information, there must be an explicit process if a generalised/unified/net result is to be formed.

Examples:

Multi source data.

Incoming data represented as:

{ "version": "1.0.0", "self": "urn:mrn:signalk:uuid:705f5f1a-efaf-44aa-9cb8-a0fd6305567c", "vessels": { "urn:mrn:signalk:uuid:705f5f1a-efaf-44aa-9cb8-a0fd6305567c": { "sources": { "ttyUSB0": { "label": "stern GPS", "type": "NMEA0183", "timestamp": "2017-05-16T05:15:50.007Z", "position": { "value": { "altitude": 0.0, "latitude": 37.81479, "longitude": -122.44880152 } }, "navigation": { "speedOverGround": { "value": 4.32693662 } } }, "ttyUSB1": { "label": "bow GPS", "type": "NMEA0183", "timestamp": "2017-05-16T05:15:50.528Z", "position": { "value": { "altitude": 0.0, "latitude": 37.81462, "longitude": -122.44880148 } }, "navigation": { "speedOverGround": { "value": 4.32482895 } } } } } } }

To yield a unified view of vessels "position" and "navigation" there must be a defined, vessel specific procedure. This procedure is an independent process that in case of this vessel subscribes to messages by "bow GPS" and "aft GPS". The process can be very simple or complex and creates another update. A super simple process might be to just copy one of the sources. A more refined one may produce the averages of both. A professional one may implement a kalman filter considering previous values and mixing them into a true state prediction. This separate process is itself considered a source.

{ "version": "1.0.0", "self": "urn:mrn:signalk:uuid:705f5f1a-efaf-44aa-9cb8-a0fd6305567c", "vessels": { "urn:mrn:signalk:uuid:705f5f1a-efaf-44aa-9cb8-a0fd6305567c": { "sources": { "avgSPG": { "label": "avgSPG", "type": "virtual", "timestamp": "2017-05-16T05:15:51.214Z", "position": { "value": { "altitude": 0.0, "latitude": 37.81471, "longitude": -122.44880150 } }, "navigation": { "speedOverGround": { "value": 4.325882785 } } } } "position": { "value": { "altitude": 0.0, "latitude": 37.81471, "longitude": -122.44880150 } }, "navigation": { "speedOverGround": { "value": 4.325882785 } } } } }

Breaking

Do away with plural: vessels -> vessel

All sources have sth they belong to. So all sources, shall be beneath its vessel, onshore site or whatever. Consequently a source identifier always contains a path : "vessel.urn:mrn:imo:mmsi:234567890.source.ttyUSB0"

Do not store foreign raw data in SignalK. No NMEA. Conversion is done before data enters the Backend by a separate process per source. Where need be that converter can track / log its activity.

Do not allow a full source definition inside another data item, just a reference, to avoid the possibility of contradicting information.

Do allow data items to be recorded below source items which inherantly defines their source. Make this the preferred way of raw data storage.