search

SignalK / specification

Signal K is a JSON-based format for storing and sharing marine data from different sources (e.g. nmea 0183, 2000, seatalk, etc)
Other
93 stars 69 forks source link

Add vessel.type, and update spec to better support data obtained from AIS. #171

Open faceless2 opened 8 years ago

faceless2 commented 8 years ago

There is currently no space in the specification for a few bits of (potentially) important information which will be obtained from other vessels over AIS. Here's a list with suggested modifications to the spec to accomodate them. $vessel means any map under "vessels".

  1. AIS type 5 (vessel static information) includes a "destination" field. This doesn't quite map to our "$vessel.navigation.destination" field, which has no space for a human-readable name, just a waypoint UUID. Suggest the addition of "$vessel.navigation.destination.name".
  2. AIS type 5 again specifies four dimensions - distance from bow, stern, port and starboard to the position reported by the GPS. This is useful information - imagine you are trying to pass behind a 300m long supertanker in fog. We already have "$vessel.details.length" and "$vessel.details.beam" - I suggest we keep these but add optional fields "$vessel.details.bowOffset" and "$vessel.details.portOffset".
  3. AIS class is useful - it gives us some information on the data that's going to be reported and the frequency of the reports (positions ever 30s for Class B, or every 10s for Class A) so we know if the data is still valid. Suggest adding a "$vessel.communication.ais.class" field which is "A" or "B" (or see last paragraph below for an alternative).
  4. There is no way to record broad "type" of vessel in the current spec. I'd argue this is useful information in general for our own vessels, but more so for others we see over AIS.

Specifically, the broad category of the AIS transmitter can be derived from the MMSI: for example any MMSI beginning with 970 is a search-and-rescue transmitter. We can also obtain more detailed vessel type information from AIS type 5 messages, which specifies one of about 100 codes including passenger vessel, hovercraft, "Tanker, Hazardous category A" (whatever that is) etc.

I would suggest using two fields for this data: "$vessel.type", which is an enum that includes at least the following values:

sart
mob
epirb
fixed
aircraft
ship (default if not specified)

While this is directly derivable from the MMSI, the sart (search-and-rescue transponder), mob (man-overboard) and EPIRB are things that you want clearly marked on your moving map, so I'd argue there's value in making this information explicitly available to a consumer. "fixed", "aircraft" and "ship" are also derivable from the MMSI prefix (ship being everything that's not one of the others, so we can make that a default if the value is missing). These values are enough to choose an icon to represent the AIS station on a map.

Second field I'd suggest is "$vessel.subtype" which is a free-form text field. This can store the detail from the AIS type 5 message for other ships, and of course being free-text you can do what you like for your own vessel (e.g. "gin palace").

Finally one variation on the above ideas is to remove "vessel.communication.ais.class" and merge it into the above - e.g. if it's a class B transponder, set $vessel.type to "small ship". The advantage is it's one less field.

faceless2 commented 8 years ago

See table 5 at http://catb.org/gpsd/AIVDM.html for details on decoding MMSI to type and country of registration.

tkurki commented 8 years ago

Thanks for your comments.

I haven't looked yet into any schema modification details, but just commenting your entry:

See also #74 about the non vessel AIS targets, maybe continue that part of discussion there?

faceless2 commented 8 years ago

Re. the bowOffset and portOffset: the idea behind this is the lat/long reported by the GPS is a point, whereas the boat is an area around that point. "length" and "beam" tell us the size of the area but you need an offset from either edge to position that boat around its lat/long. I know this isn't important on a 10m boat, but there's no reason it shouldn't be specified for the local boat if the owner wants it. The fact it's only reported over AIS doesn't mean it's an AIS-specific concept, any more than "length" or "beam" are, although I do agree they only make sense in conjunction with those attributes.

Re. subtype - OK, my use case for this is the moving map - this give us icons (perhaps sized to match the length/beam and positioned with bow/port offset as above), a different icon for each type. At a minimum I can identify it as a ship, plane, EPIRB etc. But if I have the subtype field I can customize the icon (if my map recognises the class from the text) or at the very least, roll over that icon and see more detail (it's a Tanker, MMSI X, destination Y, eta Z etc. etc.). It's detail, but it's interesting detail which is included in AIS so it seems a shame to discard it.

Currently it is AIS only, but only because it's the only method we have of sharing ship-to-ship data. If/when SignalK comes to share data between vessels, I can see this field being customized by owners with details of the type and class of their boat.

Ack on #74, will wade in when I get more time on this.

sumps commented 8 years ago

I raised this issue on Slack last year but unfortunately cannot find the original reference. As AIS is likely to be the most common source of other vessel info for the foreseeable future, it is really important that we define a schema for the boats dimensions based around the standard AIS data that you get from the Class A and Class B static data messages. Then a gateway or server can populate the key dimensions values as the relevant AIS message is received. We can expand and increase the number of boat dimensions stored in SK but I feel strongly that the simple but effective A,B,C + D dimension method utilised in AIS should be mapped one to one to a core subset of SK data objects.

image

sumps commented 8 years ago

There is a lot of useful info in the AIS Static and Voyage data messages and we should make sure that every useful field is mapped to SK.

faceless2 commented 8 years ago

There is certainly an argument for storing the fields exactly as they are reported in AIS (that is distance from bow/stern/port/starboard) - it's how they come in, after all.

For me the counter argument is this.

First, we already have length and beam defined in the model, so it's less change. They're also "natural" values, i.e. they're obvious - most boat owners could tell you what they are for their vessel. But I have no idea of the coordinates of my GPS antenna (I have several, dotted about the boat)

Second, you can convert back and forth between "four-offset" values and length/beam/port/bow values with no loss of information.

My point is if you have all four values to start with (as you would with AIS), then you can store them either way - it makes no difference. But if you've only got the length/beam values (as you would if you entered them manually) and the data model requires you to store them as bow/stern/port/starboard offsets, well then that's a pain because I haven't got the information to do that.

Incidentally, from the small print of your diagram above: If the "reference point of reported position not available but dimensions of ship are available, A=C=0 and B≠0 and D≠0" - in short, if the length/beam are all that's available you store those values in the stern and starboard offsets and set port/bow offsets to 0. This is a kludge in the AIS spec and underlines my point that length/beam are the more natural values to store.

Regarding the useful info in AIS Type 5 messags, I agree! With these suggested change that's pretty much all of them mapped. The mappings from AIS Type 5 message fields to the fields under "vessel" in the SignalK model are:

MMSI -> mmsi AIS version -> not stored IMO number -> registrations.imo Call sign -> communication.callsignvhf and/or communication.callsignhf Vessel name -> name Ship Type -> subtype (proposed) Dimension to Bow -> design.bowOffset (proposed) Dimension to Stern -> design.length.overall - design.bowOffset Dimension to Port -> design.portOffset (proposed) Dimension To Starboard -> design.beam - design.portOffset Position Fix Type -> not stored ETA month, ETA day, ETA hour, ETA minute -> navigation.destination.eta Draught -> design.draft.minimum and/or design.draft.maximum Destination -> navigation.destination.name (proposed) "Data terminal ready" -> not stored

The only fields we don't have a mapping for are the "EPFD Fix Type", which is GPS, GLONASS, Loran etc., the AIS version number (only 0 is currently defined) and "Data Terminal Ready", whatever that is.

sumps commented 8 years ago

No one likes to type in data and so I think we always need to think about these things from a user experience point of view and try to make things as automatic as possible. An AIS transponder owner is encouraged/forced to enter the data in the ABCD method and the vast majority do.There are examples of mistakes being made or the data not being entered but these are rare and one hopes will eventually be fixed by the negligent user. The vast majority of SK users will also have an AIS receiver or transponder and so will receive the dimension data in this ABCD format of all the surrounding ships and if they have a transponder also receive their boat's dimensions in this format via the VDO sentence or relevant N2K PGN. The SK server or gateway can receive the data and automatically populate the SK data fields plus any other fields such as a simple Length and Beam object. We cannot afford to lose the ABCD data though other wise no SK consumer or app that wants to display AIS vessels to scale will be able to and that is a big backwards step compared to the existing systems out there.

faceless2 commented 8 years ago

Well, first that data is only available to AIS class A transponders, and while the majority do enter it, that excerpt from your image above shows that the AIS specification authors acknowledge that this information may not be available.

I think we're at cross-purposes slightly here. I'm trying to ensure that the data from AIS and from the local boat can be stored, in the same fields.

You're right, no-one likes manual entry, but I'm talking about a one-off here. I recall at METS you mentioned the iKommunicate will have an HTML page to configure various options - presumably including boat name, mmsi, and (I would imagine) length and beam. That's almost certainly the case with all other SignalK servers too, and once set, it's unlikely to change. I presume we'd agree this data needs to be stored and represented in the model.

If so then I think there are three options available in the model which will allow us to store this locally-generated data and the data received over AIS, with no loss of information.

  1. Store length, beam and two other offsets
  2. Store four offsets
  3. Store length, beam, and all four offsets.

Option one allows you to enter length or beam manually or to receive four offsets over AIS, and store all that data in the same set of fields with no loss of information.

Option two allows you to store data received from AIS with no loss of information, but requires you to know (or guess at) the location of your GPS antenna to populate the model for your locally entered data.

Option three has redundant information. You can derive length/beam from the offsets, and derive one set of offsets from the other and the length/beam.

Anyway, that's my pitch. I just want to underline that none of these options results in any data loss. If you want to retrieve the four offsets exactly as specified in an AIS message from any of these options, you can, trivially.

sumps commented 8 years ago

Class B transponders have exactly the same ABCD dimension setup and I would recommend any gateway/server developer to simply copy the UI that the vast majority of transponder configuration programs (both A and B) use. configuration saved The market is used to this method and in my experience when you are introducing a new technology like Signal K, it is good to follow industry method and processes where possible. Therefore my vote goes to entering the four values and optionally the Length and Beam as well. My reasoning is that the AIS standard does not allow for decimal places in the ABCD dimensions so someone might want to record the actual length and beam and not be limited to entering dimensions to the nearest meter.

faceless2 commented 8 years ago

Whoops, you're correct. That data is available from Class B transponders in both Type 19 and Type 24 messages. And, from those types of message I need to add five other fields to the mapping I listed above, which are present in AIS status messages but currently unmapped in SignalK.

Vendor ID Unit Model Code Serial Number Mothership MMSI "RAIM" (Receiver Autonomous Integrity Monitoring)

I don't know how useful all of these are, but if they are to be mapped then (for the sake of starting a discussion) I'd propose "communication.ais.vendor", "communication.ais.model", "communication.ais.serial" and "communication.ais.mothership". Bit arbitrary, but all of this is very AIS specific.

The last field is part of the AIS position message (1,2 3,19), not the static data message. It is described at https://en.wikipedia.org/wiki/Receiver_autonomous_integrity_monitoring. Needs more research but I will drop some links here: https://www.ion.org/publications/abstract.cfm?articleID=4408

(edit to add notes on Type19 messages)

rob42 commented 8 years ago

I think the Vendor ID, Unit Model Code, Serial Number can best go in $source - since it is basically data that relates to the device not the communication. RAIM can go into the GPS branch, which is where it seems to be relevant

I prefer the length, beam, portOffset, bowOffset to the 4 ais fields. The result is the same, and conversion is triivial.

From a user entry perspective its a gui issue, no problem using an AIS like screen to enter the data, and simple maths to convert. But as @faceless says the length and beam are basic vessel properties useful for many things (like marina berths) so we should record them, and there is no point in duplicating info that doesnt provide any significant benefit.

rob42 commented 8 years ago

lol - is length reliable? Will owners lie about their boats length as they do about that other important measurement?

faceless2 commented 8 years ago

Apparently it's all about the beam anyway...

afischerdev commented 8 years ago

And again, let's try to bring them together.

MMSI -> mmsi (string)

Vessel - includes MOB, SART, EPIRB: IMO number -> registrations.imo Call sign -> communication.callsignvhf (callsignhf looks more amateur radio) Vessel name -> name Ship Type -> subtype (proposed integer) Navigation Status -> navigation.status (proposed integer) Dimension to Bow -> design.bowOffset (proposed integer) Dimension to Stern -> design.sternOffset (proposed integer) Dimension to Port -> design.portOffset (proposed integer) Dimension To Starboard -> design.starboardOffset (proposed integer) ETA month, ETA day, ETA hour, ETA minute -> navigation.destination.eta Draught -> design.draft.maximum (prefered before minimum) Destination -> navigation.destination.name (proposed) Text -> notifications.text (proposed)

Other object: collect in a new group aisobject with aircraft aircraft.altitude basestation aidtonavigation aidtonavigation.aidtype (integer) aidtonavigation.name

For ais objects SoG, CoG, position - if needed - from vessel

I hope a small definition, just filled in what is coming from ais message, the interpretation leave up to the client.

afischerdev commented 8 years ago

A shorter way could be

All Ais messages get a aisobject.msgtype (integer) - brings the base station, AtoN ... info to client

A text message it could go to aisobject.msgtext

aircraft.altitude is already in use as naviagation.position.altitude (java server) All other values go as usual for vessels.

samothx commented 7 years ago

I am slightly surprised that you are trying to stuff AIS data into the existing model. Would it not be easier to create a new top level signalk schema let say ais-targets and then put ais targets in there instead of populating vessels. alternately ais-targets could be coupled to a vessel as a subtype , maybe in communication. eg: vessels."urn:mrn:signalk:mmsi:4711815".communication.ais-targets or even eg: vessels."urn:mrn:signalk:mmsi:4711815".communication.ais-receivers.receiver-a.ais-targets Then you can define a clean schema following the AIS specification. I have given it a try (knowing close to nothing of schema definition) and produced the following draft: `{ "type": "object", "$schema": "http://json-schema.org/draft-04/schema#", "id": "https://signalk.github.io/specification/schemas/groups/ais-target.json#", "description": "Schema describing an AIS target as delivered by NMEA0183 !AIVDM / !AIVDO messages 1-5,18,24.", "title": "ais-target", "properties": {

# ais delivers some additions to source like
# aisType ais message number (mainly 1,2,3,4,5,18,24)
# channel ais vhf channel used for transmission A/B
# repeatInd (rather superfluous)
# also some messages carry equipment related information

# some stuff from msg 04 and 18 mainly equipment capability related
# to be defined (or ignored)

"cSUnit" : {},
"dscAttached" : {},
"bandFlag" : {},
"msg22Flag" : {},
"assigned" : {},
"raimFlag" : {},
"radioStatus" : {}

"mmsi" : {
  # more information can be drived from mmsi eg. nationality and typer of object
  "description": "MMSI number of the vessel.",
  "$ref": "definitions.json#/definitions/mmsi"
},

"mothershipMmsi" : {
  "description": "MMSI number of mothership of the vessel from msg type 24.",
  "$ref": "definitions.json#/definitions/mmsi"
},

"callSign" : {
  "type": "string",
  "description": "Callsign for VHF communication",
  "example": "ZL1234"
},

"name" : {
  "type": "string",
  "description": "The common name of the vessel",
  "example": "Motu"
},

"type" : {
  # derived from mmsi discussed in issue 171
  "type": "string",
  "description": "Type of AIS target as derived from mmsi",
  "enum": [
    "sart",
    "mob",
    "epirb",
    "fixed",
    "aircraft",
    "ship"
  ]
},

"shipType" : {
  "type": "string",
  "description": "Navigational status of Vessel",
  "enum": [
    "Wing in ground (WIG), all ships of this type",
    "Wing in ground (WIG), Hazardous category A",
    "Wing in ground (WIG), Hazardous category B",
    "Wing in ground (WIG), Hazardous category C",
    "Wing in ground (WIG), Hazardous category D",
    "Wing in ground (WIG), Reserved for future use",
    "Wing in ground (WIG), Reserved for future use",
    "Wing in ground (WIG), Reserved for future use",
    "Wing in ground (WIG), Reserved for future use",
    "Wing in ground (WIG), Reserved for future use",
    "Fishing",
    "Towing",
    "Towing: length exceeds 200m or breadth exceeds 25m",
    "Dredging or underwater ops",
    "Diving ops",
    "Military ops",
    "Sailing",
    "Pleasure Craft",
    "Reserved",
    "Reserved",
    "High speed craft (HSC), all ships of this type",
    "High speed craft (HSC), Hazardous category A",
    "High speed craft (HSC), Hazardous category B",
    "High speed craft (HSC), Hazardous category C",
    "High speed craft (HSC), Hazardous category D",
    "High speed craft (HSC), Reserved for future use",
    "High speed craft (HSC), Reserved for future use",
    "High speed craft (HSC), Reserved for future use",
    "High speed craft (HSC), Reserved for future use",
    "High speed craft (HSC), No additional information",
    "Pilot Vessel",
    "Search and Rescue vessel",
    "Tug",
    "Port Tender",
    "Anti-pollution equipment",
    "Law Enforcement",
    "Spare - Local Vessel",
    "Spare - Local Vessel",
    "Medical Transport",
    "Noncombatant ship according to RR Resolution No. 18",
    "Passenger, all ships of this type",
    "Passenger, Hazardous category A",
    "Passenger, Hazardous category B",
    "Passenger, Hazardous category C",
    "Passenger, Hazardous category D",
    "Passenger, Reserved for future use",
    "Passenger, Reserved for future use",
    "Passenger, Reserved for future use",
    "Passenger, Reserved for future use",
    "Passenger, No additional information",
    "Cargo, all ships of this type",
    "Cargo, Hazardous category A",
    "Cargo, Hazardous category B",
    "Cargo, Hazardous category C",
    "Cargo, Hazardous category D",
    "Cargo, Reserved for future use",
    "Cargo, Reserved for future use",
    "Cargo, Reserved for future use",
    "Cargo, Reserved for future use",
    "Cargo, No additional information",
    "Tanker, all ships of this type",
    "Tanker, Hazardous category A",
    "Tanker, Hazardous category B",
    "Tanker, Hazardous category C",
    "Tanker, Hazardous category D",
    "Tanker, Reserved for future use",
    "Tanker, Reserved for future use",
    "Tanker, Reserved for future use",
    "Tanker, Reserved for future use",
    "Tanker, No additional information",
    "Other Type, all ships of this type",
    "Other Type, Hazardous category A",
    "Other Type, Hazardous category B",
    "Other Type, Hazardous category C",
    "Other Type, Hazardous category D",
    "Other Type, Reserved for future use",
    "Other Type, Reserved for future use",
    "Other Type, Reserved for future use",
    "Other Type, Reserved for future use",
    "Other Type, no additional information"
  ]
},

"dimensions" : {
  "type": "object",
  "title": "dimensions",
  "description": "dimensions of target relative to GPS receiver",
  "properties": {
    "toBow": {
      "description": "distance of GPS receiver to forward extension of target",
      "type": "number",
      "units": "m"
    }
    "toStern": {
      "description": "distance of GPS receiver to aft extension of target",
      "type": "number",
      "units": "m"
    }
    "toStarboard": {
      "description": "distance of GPS receiver to starboard extension of target",
      "type": "number",
      "units": "m"
    }
    "toPort": {
      "description": "distance of GPS receiver to port extension of target",
      "type": "number",
      "units": "m"
    }
  }
},

"draught" : {
  "description": "The minimum draft of the vessel",
  "type": "number",
  "units": "m"
},

"flag": {
  "type": "string",
  "description": "The country of ship registration, or flag state as derived from mmsi",
  "example": "NZ"
},

"vendorId" : {
  "type": "string",
  "description": "Vendor id of AIS equipment from msg 24",
},

"aisVer" : {
  "type": "string",
  "description": "AIS Version",
},

"imo": {
  "type": "string",
  "description": "The IMO number of the vessel.",
  "pattern": "^IMO [0-9]{7,7}$",
  "example": "IMO 9074729",
  "maxLength": 11,
  "minLength": 11
},

"destination": {
  "title": "destination",
  "description": "The intended destination of this trip",
  "type": "object",
  "properties": {
    "name" : {
      "description": "destination port",
      "type": "string",
    }
    "eta": {
      "$ref": "../definitions.json#/definitions/timestamp"
    },
    "source": {
      "description": "Source of this data",
      "$ref": "../definitions.json#/definitions/source"
    },
    "timestamp": {
      "description": "timestamp of the last update to this data",
      "$ref": "../definitions.json#/definitions/timestamp"
    },
  }
},

"navStatus" : {
  # could be merged with vessel.navigation.state
  "type": "object",
  "description": "Navigational status of Vessel",
  "properties": {
    "source": {
      "description": "Source of this data",
      "$ref": "../definitions.json#/definitions/source"
    },
    "timestamp": {
      "description": "timestamp of the last update to this data",
      "$ref": "../definitions.json#/definitions/timestamp"
    },
    "value" : {
      "type": "string",
      "description": "Navigational status of Vessel",
      # left out several 'not available' and several 'Reserved for future use' definitions
      "enum": [
        "Under way using engine",
        "At anchor",
        "Not under command",
        "Restricted manoeuverability",
        "Constrained by her draught",
        "Moored",
        "Aground",
        "Engaged in Fishing",
        "Under way sailing",
        "Reserved for future amendment of Navigational Status for HSC",
        "Reserved for future amendment of Navigational Status for WIG",
      ]
    }
  }
},

"epfd" : {
  "type": "object",
  "description": "EPFD Fix Types",
  "properties": {
    "source": {
      "description": "Source of this data",
      "$ref": "../definitions.json#/definitions/source"
    },
    "timestamp": {
      "description": "timestamp of the last update to this data",
      "$ref": "../definitions.json#/definitions/timestamp"
    },
    "value" : {
      "type": "string",
      "description": "EPFD Fix Types",
      "enum": [
        "Undefined",
        "GPS",
        "GLONASS",
        "Combined GPS/GLONASS",
        "Loran-C",
        "Chayka",
        "Integrated navigation system",
        "Surveyed",
        "Galileo"
      ]
    }
  }
},

"time-reference" : {
  # does this need source & timestamp
  "description": "Time reference from navigational aid ais msg 4",
  "$ref": "../definitions.json#/definitions/timestamp"
},

"courseOverGroundTrue": {
  "description": "Course over ground (true)",
  "$ref": "../definitions.json#/definitions/numberValue",
  "units": "deg"
},

"headingTrue": {
  "$ref": "../definitions.json#/definitions/numberValue",
  "description": "The current true heading of the vessel",
  "units": "deg"
},

"position": {
  "description": "The position of the vessel in 2 or 3 dimensions (WGS84 datum)",
  "allOf": [{
    "$ref": "../definitions.json#/definitions/commonValueFields"
  }, {
    "$ref": "../definitions.json#/definitions/position"
  }]
},

"posAccuracy" : {
  # might have to go into positon element
  "type": "object",
  "description": "Navigational status of Vessel",
  "properties": {
    "source": {
      "description": "Source of this data",
      "$ref": "../definitions.json#/definitions/source"
    },
    "timestamp": {
      "description": "timestamp of the last update to this data",
      "$ref": "../definitions.json#/definitions/timestamp"
    },
    "value" : {
      "type": "string",
      "description": "Accuracy of the position",
      "enum": [
        "< 10m",
        "> 10m"
      ]
    }
  }
},

"utcTsSec" : {
  "type": "object",
  "description": "Navigational status of Vessel",
  "properties": {
    "source": {
      "description": "Source of this data",
      "$ref": "../definitions.json#/definitions/source"
    },
    "timestamp": {
      "description": "timestamp of the last update to this data",
      "$ref": "../definitions.json#/definitions/timestamp"
    },
    "status" : {
      "description": "Direction of turn or none as actual turn rate is not always available",
      "type": "string",
      "enum": [
          "available (default)",
          "positioning system is in manual input mode",
          "Electronic Position Fixing System operates in estimated (dead reckoning) mode",
          "positioning system is inoperative" ]
    }
    "value" : {
      "description": "second of UTC time",
      "$ref": "../definitions.json#/definitions/numberValue",
      "units": "s"
    }
  }
},

"rateOfTurn": {
  "type": "object",
  "description": "Direction and speed of turn",
  "properties": {
    "source": {
      "description": "Source of this data",
      "$ref": "../definitions.json#/definitions/source"
    },
    "timestamp": {
      "description": "timestamp of the last update to this data",
      "$ref": "../definitions.json#/definitions/timestamp"
    },
    "status" : {
      "description": "Direction of turn or none as actual turn rate is not always available",
      "type": "string",
      "enum": [
        "Starboard",
        "Port",
        "None" ]
    }
    "value" : {
      "description": "Rate of turn (+ve is change to starboard)",
      "$ref": "../definitions.json#/definitions/numberValue",
      "units": "deg/min"
    }
  }
},

"speedOverGround": {
  "$ref": "../definitions.json#/definitions/numberValue",
  "description": "Vessel speed over ground",
  "units": "kn"
},

"maneuverIndicator" : {
  "type": "object",
  "description": "Direction and speed of turn",
  "properties": {
    "source": {
      "description": "Source of this data",
      "$ref": "../definitions.json#/definitions/source"
    },
    "timestamp": {
      "description": "timestamp of the last update to this data",
      "$ref": "../definitions.json#/definitions/timestamp"
    },
    "description": "Direction of turn or none as actual turn rate is not always available",
    "type": "string",
    "enum": [
      "Not available (default)",
      "No special maneuver",
      "Special maneuver (such as regional passing arrangement)"]
  }
}

} } `

sumps commented 7 years ago

This is a good starting point but I think we need to provide a bit of history as to why we have "stuffed" AIS data in to the vessels schema. One of the key goals of Signal K is to define a schema that not only stores your own vessels data but also other vessels so that data can be shared in the cloud (ship to shore) or directly (ship to ship) so that for instance a flotilla could see exactly what conditions the lead boat was experiencing in terms of Depth, Tide, Wind, etc. or they could share tracks and waypoints.

The most common method currently of transfering data between different boats is AIS and so we have a very good basic method of populating the schema with Class A and Class B AIS data. This is defined and working very well and although I am sure there are more data fields from the Dynamic and Static AIS messages that we could "stuff" in to the vessels schema, we definitely do not want to change direction now.

What we are missing are the non-vessel AIS objects; SAR Aircraft, AtoN, SART (MOB), Base Station, etc. These are the objects that we need to define a new group for in the schema.

So the work you have done above is very good and clearly defines the different vessel related (Class A and B) data that AIS provides. I need to compare this to my own NMEA2000 AIS to Signal K mapping that I have done but not implemented yet, but as I have highlighted in #74 I think it is the non-vessel AIS types that we need to focus our attentions to.

Hope this helps provide some history and context to our discussions.

samothx commented 7 years ago

I understand the idea. Although it seems problematic to me to use AIS data in this way. Latest if you actually connect to the cloud and get other vessels information you will get collisions/inconsistencies between AIS and 'real' vessel data as well as collisions between your AIS and other vessels AIS data. Also it seems to me that the practical use of sigk today is to provide information of your vessel and feed it to gauges logs and a displays. By reducing AIS data to a compatible minimum you render the protocol useless/incomplete for AIS monitoring or force implementers to create private protocol extensions.

samothx commented 7 years ago

Another thought: Throwing away or disguising relevant data is a sin. So why don't we do something like I suggested - supply a complete set of vessel specific AIS data (mapped to vessels.id.communication.ais.receiver.vessel or so) and then Server implementers or even server users (by configuration) can choose if they want to have AIS vessels show up in vessels additionally with a vessel compatible set of values. Maybe put a type flag on them to distinguish them from real vessels. This way you do not have to extend the vessel schema to something it isn't meant to be and at the same time you do not throw away precious data - at the price of a little bit of duplicate data in the model..

samothx commented 7 years ago

Btw my schema is missing all the AtoN stuff...

sumps commented 7 years ago

I think you have mis-understood me. I am certainly not suggesting that we throw away any AIS data, my comment "we definitely do not want to change direction now" meant we do not want to bolt on a separate AIS group and extract all of the Class A and Class B data that we currently store in the vessels schema and place it in the new group.

We need to look at all of the AIS Vessel data and continue to implement this in the vessel schema and in parallel look at the non-vessel AIS data and see how this can best be added to the schema in a group at the same level as vessels - something like "Marine Objects" that are objects unique to the marine world. In theory if the SIgnal K schema was ever expanded for applications outside of marine, then we could have other application specific groups like "Aviation Objects", "Road Objects", etc.

faceless2 commented 7 years ago

It's also worth noting the model is designed to handle data from different sources, which may conflict. The usual example is two GPS units on board, but there's no reason this couldn't extend to two reports of another ships position, one from AIS and one from the cloud (or wherever).

As for "throwing away data is a sin", I would suggest the greater sin is basing your abstract data model on a particular implementation ;-) AIS is just a means - potentially one of several - of obtaining data about ships position, course and so on. What we're interested in is the logical value of what those fields represent, not how they're obtained.

In fact I'd argue that this distinction is fundamental to SignalK, and what allows it to combine data from different sources into one coherent model.

sumps commented 7 years ago

Here is an example of what iKommunicate is typically converting from an AIS;

{
    "context": "vessels.urn:mrn:imo:mmsi:235097602",
    "updates": [{
        "source": {
            "type": "NMEA2000",
            "src": "003",
            "pgn": 129039,
            "label": "N2000-01.003"
        },
        "timestamp": "2010-01-01T00:49:14Z",
        "values": [{
            "path": "navigation.position",
            "value": {
                "longitude": -1.149100,
                "latitude": 50.761833
            }
        }, {
            "path": "navigation.speedOverGround",
            "value": 1.899994
        }, {
            "path": "navigation.courseOverGround",
            "value": 4.621604
        }]
    }]
} {
    "context": "vessels.urn:mrn:imo:mmsi:235002514",
    "updates": [{
        "source": {
            "type": "NMEA2000",
            "src": "003",
            "pgn": 129038,
            "label": "N2000-01.003"
        },
        "timestamp": "2010-01-01T00:49:14Z",
        "values": [{
            "path": "navigation.position",
            "value": {
                "longitude": -1.107667,
                "latitude": 50.792999
            }
        }, {
            "path": "navigation.headingMagnetic",
            "value": 1.588201
        }, {
            "path": "navigation.speedOverGround",
            "value": 0.509998
        }, {
            "path": "navigation.courseOverGround",
            "value": 1.570801
        }]
    }]
} {
    "context": "vessels.urn:mrn:imo:mmsi:235074703",
    "updates": [{
        "source": {
            "type": "NMEA2000",
            "src": "003",
            "pgn": 129038,
            "label": "N2000-01.003"
        },
        "timestamp": "2010-01-01T00:49:14Z",
        "values": [{
            "path": "navigation.position",
            "value": {
                "longitude": -1.107683,
                "latitude": 50.804520
            }
        }, {
            "path": "navigation.speedOverGround",
            "value": 0.000000
        }, {
            "path": "navigation.courseOverGround",
            "value": 3.000202
        }]
    }]
} {
    "context": "vessels.urn:mrn:imo:mmsi:235899912",
    "updates": [{
        "source": {
            "type": "NMEA2000",
            "src": "003",
            "pgn": 129039,
            "label": "N2000-01.003"
        },
        "timestamp": "2010-01-01T00:49:14Z",
        "values": [{
            "path": "navigation.position",
            "value": {
                "longitude": -1.134580,
                "latitude": 50.855320
            }
        }, {
            "path": "navigation.speedOverGround",
            "value": 0.050000
        }, {
            "path": "navigation.courseOverGround",
            "value": 5.297105
        }]
    }]
} {
    "context": "vessels",
    "updates": []
} {
    "context": "vessels.urn:mrn:imo:mmsi:235013829",
    "updates": [{
        "source": {
            "type": "NMEA2000",
            "src": "003",
            "pgn": 129038,
            "label": "N2000-01.003"
        },
        "timestamp": "2010-01-01T00:49:15Z",
        "values": [{
            "path": "navigation.position",
            "value": {
                "longitude": -1.114400,
                "latitude": 50.791264
            }
        }, {
            "path": "navigation.speedOverGround",
            "value": 0.000000
        }, {
            "path": "navigation.courseOverGround",
            "value": 5.979505
        }]
    }]
} {
    "context": "vessels.urn:mrn:imo:mmsi:235074416",
    "updates": [{
        "source": {
            "type": "NMEA2000",
            "src": "003",
            "pgn": 129794,
            "label": "N2000-01.003"
        },
        "timestamp": "2010-01-01T00:49:15Z",
        "values": [{
            "path": "/",
            "value": {
                "name": "SD SUZANNE"
            },
            "path": "communication",
            "callsignVhf": "2COB9",
            "value": {}
        }]
    }]
}

Like I said previously, we need to continue to do more work to convert more AIS Vessel data, but it is working and providing good data in to the current schema.

samothx commented 7 years ago

Ok, I am getting the message.. But I cannot agree. AIS is not just some means of getting information. If you look at cruisers you will see that almost everyone is using it excessively and relying on it when sailing open ocean at night. Same when you are in high traffic areas in reduced visibiltity. As a sailor I do not just want to know that something is moving (or not) out there, I want to know is it sailing (for reasons of right of way) is it trawling and so on. For many it is one of the most important pieces of information they are getting from their vessels systems. Currently signalK implementations are only providing a subset of the available information. And that is not good because they are already being marketed in consumer products. @sumps looking at your sigk delta you provide the imo and callsign on top of what the reference node server provides. On the other hand you provide headingMagnetic where my docs (and signalk-server-node) say (and I don't have the official ITU docs) that the heading provided is true.
In my opinion an AIS vessel is different from a real vessel and has to be treated differently. What do you want to do, should you ever get radar targets into signalk ? Make vessels out of them ? Currently being in NZ I would put it so: It is like comparing cumara and potatoes. They have some things in common but a lot of things are different and you want to be aware of that when you make a meal out of them. So far in signalk you are putting them both in one pot and don't even supply a means to distinguish between them - except maybe that everything that is not self is AIS. That you do it to populate the sadly empty vessels compartment is not acceptable really. I guess it has something to to with existing codebase too. Anyway, I am just a passer by looking into signalk. I don't want to upset anybody so I will come back and look when schema v2 is out and mind my own business meanwhile.

faceless2 commented 7 years ago

I am 99% sure AIS vessel "navigation status" (i.e sailing, trawling) is stored, somewhere, although it's late here and I don't fancy wading through the spec (which I am not up-to-date on). I opened this issue to list fields that weren't stored in the model, and I didn't put it on that list. I agree it's extremely useful.

Really, I think the only point I differ with you on is that these should be put into a separate top level "ais-target" object. You're right, it's all going in one "pot", called vessels. I like that, myself.

Here's a contrived, over-simplified and entirely hypothetical example: you're out sailing a club regatta and the boats around you, as well as transmitting AIS position, are also sending info on sail plan, apparent wind, or something which we're interested in over 3G to the internet. We're doing the same and are also downloading their information. How do we store information on other boats from these two sources, AIS and the net?

Do you store it as "ais.otherboat.position" and "internet.otherboat.position", or do you store it as "vessels.otherboat.position[ais]" and "vessels.otherboat.position[internet]"?

SignalK has chosen the second option - it's not that the other option is invalid, but the second option has several advantages.

Again it's worth underlining that no data is lost here: check out the bit on the spec regarding multiple sources. And it's also worth underlining that if useful information from AIS is not being currently stored, then I agree it should be: that's why I opened this issue.

samothx commented 7 years ago

@faceless2 there is a vessel state in vessel.navigation.state. To answer your question. If you have AIS yourself you would probably not subscribe to other boats AIS. If you did I would think it would be found in the other boats vessel information vessels.id.communication.ais.targets or so.

In your regatta example what are the individual boats sending to the server? Only their own information or all vessels ? If they are sending all, the server will have plenty to do because complexity of that merge must be O^2 of the number of sigk sending vessels. E.g 20 vessels makes 400 vessel informations to be sent processed and merged at every update. Quite a waste of bandwidth and computing power specially because the vessel infos mightl be full of all kinds of data, not just AIS. If every Vessel sends just its own self then all vessels without AIS do not see any AIS information at all. A pitty, borrowing AIS from other vessels would be great. If every vessel equipped with AIS has its own little AIS folder there is at least a choice. The server or individual vessels without AIS could detect the presence of AIS data in other vessels and subscribe to the AIS delta of just one vessel. By the way in a race vessels probably would not really like to share where they are and what the conditions are there. Finding the best breeze is what makes a good race skipper.. They just did it in the Vendee Globe but I guess someone paid a lot of money for that.

sumps commented 7 years ago

@samothx these issue discussions can sometimes feel like an arguement and it is easy to take them personally but after a while you realise that everyone is trying to collaborate and come up with the best solution. I was really impressed with the work you did and the last thing I want to do is put you off contributing, but equally we cannot just accept everyone's proposal and keep making large changes to the schema unless there are really good reasons to do so. There are some corner stones to the Signal K design, established right at the beginning, that we have to maintain and unfortunately your proposal to strip out the AIS Vessel data from the vessels schema goes against one of these and that is why @faceless2 and myself have been trying to explain the way it is done currently and the benefits this provides. Part of the problem, is that the current AIS implementation is not complete and as you rightly point out only a subset is currently supported. This is not right, but being an Open Source project with limited contributors, this is a reality of how these projects evolve. It would be great if you could help us turn the subset in to a complete implementation and then I think you would see that the AIS data is not lost and can be used very well within the vessel schema as we have it now. I have dug out my notes on the mapping of AIS objects to Signal K and included them in this spreadsheet for you to look at. I have also attached the ITU doc on AIS. It would be great if you could look them over and see how I am proposing to get more AIS data in to Signal K and point out any data that I have missed. AIS Mapping to Signal K.xlsx ITU-R M.1371-5.pdf

samothx commented 7 years ago

@sumps thanks for your praise. In turn I will praise you for your management qualities, Ok ? And thanks for the ITU docs, it is the first time I get my hands on those. I have just ported an AIS parser I coded in 2014 from java to javascript so I am pretty much at home in the NMEA0183 side of things. Unfortunately (I haven't checked against your official docs) a lot of things are not very straightforward there. For instance something as simple as SpeedOverGround has nasty rules: Values up to 1022 are Speed over ground in kn * 10. 1022 means speeds higher than 101.1 kn, 1023 means value not available. Rate of Turn is just as awkwardly coded. I did the coding for most of my java AIS parser on the boat in Panama. That is better than working with AISHub data because there is an incredible amount of traffic and AIS repeaters so you actually see the traffic through the whole canal from Colon to Panama. Looking at that data most vessels do not event send a rate of turn. They send 127, 0 or -127 meaning left, straight ahead or right. Few vessels actually measure rate of turn. AIS is byte crunching at its worst. Such values will be hard to reflect correctly in sigk without making some awkward changes. Also a lot of fields have 'not available' codes. There has to be a strategy to handle those. Does 'not available' delete a previously set field ? Does AIS have lower priority and cannot delete a SOG value say from the GPS ? Btw. these are some of the reasons why a separate AIS schema could make life a lot easier.

As to your spreadsheet. It would make life easier to take a bit of the complexity out of it. What I mean is it is not important what message generates a parameter, you just want to know which parameters can occur. Now you add a description to each of the parameters and an interpretation / possible values / meanings and decide if you need to transfer it to signalk or if it is unimportant enough to ignore. For all those that are not ignorable you will have to find a home in signalk. I can deliver a list from my parser containing all (supported) parameters in NMEA0183 AIS. In the Parser I have chosen to ignore all msg types other than 1,2,3,4,5,18,19,21,24.In panama I saw messages 8,17,20 but they are related to communication and rather rare. I had to go to AISHub to actuall see a message 24.
I can start a document that contains those parameters and qualify them. I wouldd add another sheet to th your spreadsheet and gather ITU descriptions from your sheets, add possible values and meanings and also add what you have suggested of in terms of sigk mapping. That might be a good base for discussion then.

samothx commented 7 years ago

Ok, here is the data available from my point of view. I will look through the ITU docs and find out if my docs are leaving stuff out or telling the wrong thing. Also I am only looking at messages 1,2,3,4,5,18,19,21,24. There is some interesting data in some other messages but it is really none of our business because it is mostly ship to shore communcation. It is also probably hell to decode and a pain to store in signalk. You also probably won't find a parser that will do those messages.. I hope I haven't spoilt your document. I am on Linux and working with Libreoffice.. AIS.Mapping.to.Signal.K.xlsx

tkurki commented 7 years ago

If we want to to capture all the details of the AIS data why not define a data model/api that carries all the AIS data, a one to one mapping from ITU spec (or in practice NMEA messages) to JSON?

echo '2014-08-15T19:00:00.443,4,129038,43,255,27,01,10,dc,6d,0f,d0,9f,63,0e,e2,26,9d,23,ed,a6,ad,83,02,0c,80,08,88,ae,0a,00,00,fe' | analyzer -json | json
INFO 2017-02-18T08:04:03.900Z [analyzer] New PGN 129038 for device 43 (heap 9442 bytes)
{
  "timestamp": "2014-08-15T19:00:00.443",
  "prio": 4,
  "src": 43,
  "dst": 255,
  "pgn": 129038,
  "description": "AIS Class A Position Report",
  "fields": {
    "Message ID": 1,
    "Repeat Indicator": "Initial",
    "User ID": 258858000,
    "Longitude": 24.141,
    "Latitude": 59.7501666,
    "Position Accuracy": "High",
    "RAIM": "not in use",
    "Time Stamp": "59",
    "COG": 254.7,
    "SOG": 6.43,
    "Communication State": "32780",
    "AIS Transceiver information": "Channel B VDL reception",
    "Heading": 256,
    "Rate of Turn": 0.02,
    "Nav Status": "Under way using engine",
    "Regional Application": 0
  }
}

This discussion has again bumped against the idea that all the data items must have a single home path in Signal K. I think that modeling things as a single JSON document is starting to be a curse.

If you instead think of Signal K as an HTTP API there is nothing technical stopping us from making the same ais data available under /vessels as separate vessels with mmsi, name, position, speed etc (current version) and having it under /vessels/self/communication/ais in AIS oriented format.

Signal K is also a streaming format. There the delta message consists essentially of

This is not entirely true, since the context in delta currently holds the vessels keyword, but we need to figure out a nice way to include AtoNs, aircraft and whatnot in delta as well.

The hard part is going for uniformness in SK implementations: every SK device should be following the same rules and adding multiple ways to access the same data just adds more room for differing implementations and spec coverage. We already have the streaming subscription feature, which is a bit complex to implement and a straightforward gateway (think iKommunicate) may not have that, whereas the reference implementations should naturally implement everything.

tkurki commented 7 years ago

In your regatta example what are the individual boats sending to the server? Only their own information or all vessels?

Could be either, depending on the server and available bandwidth. I assume by all vessels you mean AIS data.

If they are sending all, the server will have plenty to do because complexity of that merge must be O^2 of the number of sigk sending vessels.

The server may

Point is that this is that we do not know, so let's not use that as an argument.

If every vessel equipped with AIS has its own little AIS folder there is at least a choice.

That is not the only way to solve this - take a look at delta structure. In essence you would like to subscribe to data by

BTW in this fleet example subscribing strictly by a single source it not the most robust way to do it, as

By the way in a race vessels probably would not really like to share where they are and what the conditions are there.

Take away the racing there and think of group of boats, maybe on a leisurely cruise around the Baltic or just some random group of friends out there, and your condition doesn't hold. It doesn't hold for all racing either: AIS transponders are also becoming mandatory in some races.

sumps commented 7 years ago

If we had a group called marineObjects at the same level in the schema as vessels, this would work. All non-vessel AIS objects still have an MMSI number so can have the same Unique ID format as vessels and if other non-AIS marineObjects are added in the future they just have a UUID. @samothx to answer your question if data is invalid or not available we send a null in the JSON value.

afischerdev commented 7 years ago

I tried to make some update definitions for the missed AIS messages. Some ot the fields are not defined at the moment. E.g. I used sumps idea of 'marineObjects' for not vessel objects and added a group 'aisobject' to get a place for the not available items.

Other AIS messages 4: fixed station

{
    "context": "marineObjects.urn:mrn:imo:mmsi:123456789",
    "updates": [{
        "timestamp": "2010-01-01T00:49:15Z",
        "values": [{
            "path": "aisobject",
            "value": {
                "msgtype": 4
            },
            "path": "navigation.position",
            "value": {
                "longitude": -1.114400,
                "latitude": 50.791264
            }
        }]
    }]
}

5: Static and Voyage Related Data

{
    "context": "vessels.urn:mrn:imo:mmsi:123456789",
    "updates": [{
        "timestamp": "2010-01-01T00:49:15Z",
        "values": [{
            "path": "/",
            "value": {
                "name": "SD SUZANNE"
            },
            "path": "registrations",
            "value": {
                "imo": "12345"          
            },
            "path": "communication",
            "value": {
                "callsignVhf": "2COB9"

            },
            "path": "aisobject",
            "values": {
                "shiptype": 36
            },
            "path": "navigation.destination",
            "value": {
                "waypoint": "London"

            },
            "path": "navigation.destination",
            "value": {
                "eta": "2017-03-02T12:00Z"

            },
            "path": "design.draft",
            "value": {
                "maximum ": 3

            },
            "path": "sensors.gps",
            "value": {
                "fromBow": 8

            },
            "path": "sensors.gps",
            "value": {
                "fromCenter": 2

            },
            "path": "design.length",
            "value": {
                "overall": 31

            },
            "path": "design",
            "value": {
                "beam": 5

            }       
        }]
    }]
}

9: SAR Aircraft

{
    "context": "marineObjects.urn:mrn:imo:mmsi:123456789",
    "updates": [{
        "timestamp": "2010-01-01T00:49:15Z",
        "values": [{
            "path": "aisobject",
            "value": {
                "msgtype": 9
            },
            "path": "navigation.position",
            "value": {
                "longitude": -1.114400,
                "latitude": 50.791264,
                "altitude": 345
            },
            {
            "path": "navigation.speedOverGround",
            "value": 123.45
            },
            {
            "path": "navigation.courseOverGround",
            "value": 5.979505
            }
        }]
    }]
}

14: text messages

{
    "context": "marineObjects.urn:mrn:imo:mmsi:123456789",
    "updates": [{
        "timestamp": "2010-01-01T00:49:15Z",
        "values": [{
            "path": "aisobject",
            "values": {
                "msgtype": 14,
                "msgtext": "A message text"
            }
        }]
    }]
}

21: AtoN

{
    "context": "marineObjects.urn:mrn:imo:mmsi:123456789",
    "updates": [{
        "timestamp": "2010-01-01T00:49:15Z",
        "values": [{
            "path": "aisobject",
            "values": {
                "msgtype": 21,
                "aidtype": 6
            },
            "path": "/",
            "value": {
                "name": "AtoN light"
            }.
            "path": "navigation.position",
            "value": {
                "longitude": -1.114400,
                "latitude": 50.791264
            }

        }]
    }]
}

1: MOB, EPIRB, SART

{
    "context": "marineObjects.urn:mrn:imo:mmsi:972016789",
    "updates": [{
        "timestamp": "2010-01-01T00:49:15Z",
        "values": [{
            "path": "aisobject",
            "values": {
                "msgtype": 1,
                "navstatus": 14
            },
            "path": "navigation.position",
            "value": {
                "longitude": -1.114400,
                "latitude": 50.791264
            }

        }]
    }]
}

At the end we 'only' have a new root object and a new group with fields for msgtype, navstatus, shiptype, aidtype, msgtext

PS. I'm standby to place something like this into the signalk-parser-nmea0183 libs. ggencode is already changed to bring the data into signalk.

rob42 commented 7 years ago

Been reading thru the Issue-I may have missed a bit as I am on my cell phone-but my take is as follows. @Samothx -good work, both solutions and challenging our thinking. If we cant defend a position then we should revisit it! But as the others have mentioned there are reasons to avoid a separate AIS group.

The primary one for me is to allow filtering of data, and application of security etc. I think of it like a Linux filesystem , Sym links and all. So I can filter keys (files) by wildcard a regex etc, and only receive a manageable dataset and I can control access to info easily. . That gets much more complex when the same data is scattered between structures. (vessels and Als)

Also if we are missing any useful data, lets add it. That one of signal Ks design goals-being extensible

rob42 commented 7 years ago

we do need other root paths for shore facilities,SART, aircraft, even cars etc. We have discussed this before, I think the logical root of the tree is org.signalk then either marine.vessels, or just vessels. We dont have a good name for other objects root yet. Shore, or static maybe, I can see a case for others, virtual, regulatory, etc

samothx commented 7 years ago

Just noticed I uploaded the original Document not my updated version further up. So here is the Document including a list of Parameters in sheet Parameters and the list of ShipTypes/Cargo in ShipTypes and cargo sheet so we know what we are talking about. I added a column qualifying the parameter as Must Have / Nite to Have / Not needed too. Thats just what I think...
AIS.Mapping.to.Signal.K.xlsx

afischerdev commented 7 years ago

@rob42 I agree that 'marineObjects' as a group is a little crude. 'marineObjects' as root would be correct, but seeing org.signalk as root is very smart and hits the core of the project.

It would therefore have to be generated parallel to vessels new objects, which can then have properties such as navigation, design or so

@samothx Good summary and a great basis for further discussions. I have some change requests:

samothx commented 7 years ago

I put the spreadsheet into a Google Document. For viewing go to https://docs.google.com/spreadsheets/d/1wZTlvLNAN4GrXk3TfEtJne-76pmzCgBVRd_HMQfcwSQ/edit?usp=sharing If you want to edit, PM me your email on slack.. I also colored it in some more. Green means it has a place in the schema, yellow means it sort of has a place but there is a problem, Orange means i think it should have a place but I can't see one. Grey is I don't think it seriously needs a place in signalk

rob42 commented 7 years ago

Excellent. jez - ais is a good example how to write a horrible protocol :-( Some notes, I cant edit the spreadsheet so I cant add directly. 1 aisType logically goes in source, we add an optional param, like we have for N2K 9 posAccuracy - map to gnss.methodQuality 0="GNSS Fix", 1="DGNSS fix" 10 navStatus - add missing options to navigation.state enum 17 epfd - this is not the same as 9 above - need a new key gnss.type? 22 shipType - add design.type in design, or aisType in vessels - ?? 23-26 - there are current keys in design for vessel loa and beam, and we have sensors for a particular sensors offset. These can be mapped and all 4 ais dimensions calculated, or the signalk keys can be reversed out of a message. The UI presentation is up to the implementation. 32 destination - navigation.destination.waypoint is a location, and ais is just random text. We should add an optional commonName attribute to the navigation.destination object. Implementations can offer waypoint lookup if they want. 34 sog - map HIGH to 102.2 as magic number - maybe add warning in meta? We leave it to users or server to take further action? Since source will show ais, thats workable if ugly, probably a necessary evil of supporting these horrible protocols. Also not many vessels at 100+ knots. 35 mothershipMmsi - add to vessels 40 rot - dont have this, my boats composite :-) RIGHT and LEFT can be handled same as for 34. 41-42 map these to new entries in a new tree for marine objects that dont move. navigationAids maybe? There are e-libraries of this data available, once populated it could display on charts, effectively updating older charts? A smart server could look for and warn about them. 45 would seem to need a new attribute in navigation

samothx commented 7 years ago

Just for the fun of it. I have got the values that have a home in signalk sorted on my server Added

afischerdev commented 7 years ago

@rob42 Please no HIGH definition for sog. For airplane it is 1022, just the decimal point is dropped.

I recommend to change the labels for Signal-K in spreadsheet. All definitions starts with vessel/mmsi. Isn't it better to define e.g. object/mmsi?

And we have to design the new objects for AidToNavigation BaseStation Airplane MOB SART EPIRB

Bring AToN and base station together to a 'Fixed Station'? Or is it not fixed when it is a cardinal mark?

sumps commented 7 years ago

An AIS Base Station and an AIS AtoN are two completely different objects and I would not mix them under any circumstances. I think you will find that AtoNs will be much more than just a cardinal buoy, as Marine Authorities start to evaluate/understand them and find new uses for them.

sumps commented 7 years ago

Also Airplane is too generic and limiting if Signal K starts to be used outside of Marine. It needs to be SARAircraft "Search and Rescue", which is a specific type of transponder that only SAR aircraft can fit.

sumps commented 7 years ago

Sorry last comment, I would prefix all of these objects with AIS;

aisAtoN aisBaseStation aisSARaircraft aisMOB aisSART aisEPIRB

This then clearly defines where this data has originated and also differentiates for instance between an AIS MOB and another type of MOB system, of which there are many.

afischerdev commented 7 years ago

Oh, yes I know. I've been to the Adriatic Sea last year. Here I found that AToNs provides you with all other AToN around the coast of Adriatic.

New list makes it clearer.

rob42 commented 7 years ago

@sumps The idea behind the HIGH definition is not about limiting the signalk value. No reason not to have the full/accurate sog in signalk. But when we receive a HIGH value from an AIS sentence, we need to put some numeric value into signalk. So my idea is to put 102.2 for a vessel, and probably 1022 for an aircraft, and a warning in meta that these are AIS HIGH entries, and the real sog is unknown but higher.

rob42 commented 7 years ago

@sumps I'm keen to see all the ais* data in signalk, but there is no reason to follow the ais naming scheme or categories. Mostly the whole AIS message structure is a horrible kludge to be able to send the maximum amount of data over VHF marine radio frequencies. Thats why convoluted sog and 6-bit values, gets the most out of a very short digital message. We dont have that problem, so we can design it on a more rational basis.

One of the 'unplanned' outcomes of concentrating on our boats early on is the vessels tree. It made sense at the time, as that was all we had to deal with. But now we have other classes of object, and need a suitable branch to put them.

Easy enough, add a sibling to vessels, we already have sources and resources, but we need to name them properly so we dont do the same as we did with vessels.

Reading up about AtoN's (we dont have any here) it seems they fit into resources which we have defined as "Resources to aid in navigation and operation of the vessel including waypoints, routes, notes, etc."

That still leaves others. aircraft is a reasonable parallel to vessels, and also works with vehicles etc in future. All these objects have the fundamental property of moving. A SAR vessel is no different from a normal vessel , we add SAR keys to vessels. Same for SAR aircraft

MOB we already have. Since MOB is a function of a vessel (the one you fell off) signalk naturally propagates this across other vessels - so you can see vessel x just lost a person overboard.

So then we need definitions for objects that dont (normally) move. I am avoiding naming each, as we will end up with a thousand categories, we need some parent key.

I propose shore for all land based objects, that would include base stations, repair facilites, marinas, etc - eg anything that you can walk to with dry feet. Inside that we add subkeys to organise them.

I think we also need another category for fixed objects at sea, vessels doesnt really cut it for oil rigs etc. Cant think of a decent name, maybe fixed, or permanent?

webmasterkai commented 7 years ago

Building an inheritance style taxonomy tree is an extremely difficult process that I hope SK avoids entirely. The real value of SK is about defining how different properties are communicated.

Where in the tree does a seaplane fit? Or an amphibious vehicle? A person overboard could be from a boat, or an airplane, or any number of various circumstances.

I think the place to focus energy is on how SK describes the various attributes and properties instead of what tree they live in. NMEA 0183 defined a way of sending GPS coordinates. It does not care what the thing that has GPS coordinates is. NMEA 0183 is used on so many different GPS receivers not used in the marine industry.

The use of vessels should probably just be replaced with something more generic like entity. Give the entity a UUID and start describing it. Let the attributes describe what the thing is, not where it lives in a taxonomy tree.

I want to use SK in an RV or drone or off-grid cabin that has a gps coordinate, solar panels, weather station etc.

webmasterkai commented 7 years ago

I think of AIS as a particular type of sensor that is sending out property values like any other sensor system might. For example many sensors have a max/high/over value where it can not read any higher. It would probably be good for SK to have a way of saying the sensor/source is over its limit.

rob42 commented 7 years ago

@webmasterkai signalk can already handle the max/high/over scenarios using meta and notifications. That method allows for any misc event or state, not just max/high/over so I dont see the need for a flag or enum for every key. The idea behind notifications andmeta` is to make it a single generic way to deal with all events, so that a client can easily monitor anything its interested in.