Are sources working right?

[sailoog]

Maybe I'm misunderstanding how sources work but I'm getting strange behaviours... First of all this is my settings file:

{
    "vessel" : {
        "mmsi" : "000000000",
        "uuid" : "urn:mrn:imo:mmsi:000000000"
    },
    "pipedProviders" : [{
            "pipeElements" : [{
                    "type" : "providers/udp",
                    "options" : {
                        "port" : "55556"
                    }
                }, {
                    "type" : "providers/liner"
                }, {
                    "type" : "providers/from_json"
                }
            ],
            "id" : "I2Csensors"
        }, {
            "pipeElements" : [{
                    "type" : "providers/udp",
                    "options" : {
                        "port" : "55557"
                    }
                }, {
                    "type" : "providers/liner"
                }, {
                    "type" : "providers/from_json"
                }
            ],
            "id" : "1Wsensors"
        }
    ],
    "interfaces" : {}
}

I have connected I2C sensors and 1W sensors to the GPIO of a raspberry 3 and all is working right. I send delta format to the providers with this format:

{
  "context": "vessels.urn:mrn:imo:mmsi:000000000",
  "updates": [
    {
      "source": {
        "type": "1W",
        "src": "0316013faeff"
      },
      "timestamp": "2016-07-28T18:18:46.074Z",
      "values": [
        {
          "path": "propulsion.engine1.temperature",
          "value": 301.837
        }
      ]
    }
  ]
}

This is what I'm getting in full format:

...
{"propulsion": {
  "engine1": {
    "temperature": {
      "value": 301.837,
      "$source": "1Wsensors.0316013faeff",
      "timestamp": "2016-07-28T18:18:46.074Z"
    }
  }
}
}
...
{"environment": {
  "outside": {
    "pressure": {
      "value": 101365.997314,
      "$source": "I2Csensors.BMP180",
      "timestamp": "2016-07-28T18:18:46.297Z"
    },
    "temperature": {
      "value": 302.25,
      "$source": "I2Csensors.BMP180",
      "timestamp": "2016-07-28T18:18:46.297Z"
    }
  },
  "inside": {
    "humidity": {
      "value": 58.1872024536,
      "$source": "I2Csensors.HTU21D",
      "timestamp": "2016-07-28T18:18:46.297Z"
    },
    "temperature": {
      "value": 302.68,
      "$source": "I2Csensors.HTU21D",
      "timestamp": "2016-07-28T18:18:46.297Z"
    }
  }
}
}
...

As you can see, sources have this format:

$source:"<provider id>.<sensor id>"
$source":"1Wsensors.0316013faeff"
$source":"I2Csensors.BMP180"
$source":"I2Csensors.HTU21D"

But I would like to have only one provider in my settings file with "id" : "sensors" and to have this format in all sources:

$source:"<provider id>.<type>.<sensor id>"
$source":"sensors.1W.0316013faeff"
$source":"sensors.I2C.BMP180"
$source":"sensors.I2C.HTU21D"

According to Signal K definition, sources can contain this properties: label, type, src, pgn, sentence, talker, +any string. Being required label and type. And this is my problem, whatever I put in the delta message when I send to signal k server, only the src property affects the $source path. With the rest of properties I only get:

$source:"<provider id>.XX"
$source":"sensors.XX"

Is it possible that signalk-server-node is only working for NMEA 0183/2000 data and not for custom data?

[sailoog]

I have also tried to manage sources creating a tree vessels.self.sources as suggested in the documentation to deal with multiple values, so I send the delta format:

{
  "context": "vessels.urn:mrn:imo:mmsi:000000000",
  "updates": [
    {
      "source": "vessels.urn:mrn:imo:mmsi:000000000.sources.sensors.1W.0316013faeff",
      "timestamp": "2016-07-29T09:49:06.463Z",
      "values": [
        {
          "path": "propulsion.engine1.temperature",
          "value": 301.15
        }
      ]
    },
    {
      "source": "unknown",
      "timestamp": "2016-07-29T09:49:06.463Z",
      "values": [
        {
          "path": "sources.sensors.1W.0316013faeff",
          "value": 301.15
        }
      ]
    }
  ]
}

But this is what I'm getting in full format:

  "propulsion": {
    "engine1": {
      "temperature": {
        "value": 301.275,
        "$source": "undefined.XX",
        "timestamp": "2016-07-29T10:06:53.393Z"
      }
    }
  },
  "sources": {
    "sensors": {
      "1W": {
        "0316013faeff": {
          "value": 301.275,
          "$source": "undefined.XX",
          "timestamp": "2016-07-29T10:06:53.393Z"
        }
      }
    }
  }

I'm lost here :(

[tkurki]

TL;DR:

Non-nmea source handling is incomplete and should be fixed.

When talking about sources there are three things to consider:

• source in delta
• $source "reference" in the full tree
• sources subtree in the full tree

Sources subtree

The idea in the sources subtree is that there is a place where you can have metadata about a sensor/source. The subtree is a two level hierarchy. The first level is for identifying the "top level source": for example if there are multiple n2k buses onboard there could be "n2k-1" and "n2k-2" or in case of multiple NMEA0183 input connections "plotter", "windinstruments" and "ais". A real world sensor example from my own boat I have "i2c" provider in the configuration file, which runs a custom JavaScript provider that outputs sources as "i2c.housevoltage" and "i2c.housecurrent".

Then under the "top level source" there should be enough information to identify the source within that category/connection/bus. For N2K it is the NMEA2000 source transmitted in the pgn. For NMEA0183 it is the talker id of the sentence. For non-NMEA sources it is not really well defined. Looking at the code the label in the delta source is used as the top level source, but no other fields are used. So in the node server under /sources there is only one level, derived from the label property the delta source.

However your examples are not about the sources subtree but the $source associated with the values. It should match the /sources subtree, but I don't think the implementation is correct. For details see the relevant code. It actually doesn't handle non-NMEA sources at all, it just handles N2K and assumes the rest is NMEA0183, this is where you get the undefined.XX.

This should be fixed.

As for your example:

$source:"<provider id>.<type>.<sensor id>"
$source":"sensors.1W.0316013faeff"
$source":"sensors.I2C.BMP180"
$source":"sensors.I2C.HTU21D"

is there a specific reason for the sensors level in this? I am about to add some 1W sensors to my own Pi and then possibly a second I2C connection. To me the first level in the $source (that must correspond to the first level under /sources) should identify the bus/input and the second the transmitting entity within that connection.

I2C and 1W would map pretty well to this model. However multi-sensor boards with a single I2C address would warrant another level underneath though. My own electricity meter is a single I2C ADC board with two outputs. Therefore timestamps as associated in the two level hierarchy's leaf will show just the last timestamp for any data from this bus member.

This is actually not about provider ids at all. As far as I can recall node server handles deltas from any source and doesn't really care about the provider ids at all (except that having duplicates will probably result in the earlier one being ignored, but that happens when the JSON is parsed and is not really a design decision).

So instead how about

$source:"<bus id>.<type>.<sensor id>"
$source":"1W.0316013faeff"
$source":"I2C-0.BMP180"
$source":"I2C-1.HTU21D"

for a system with one 1W and two I2C connections and three sensors.

BTW I would appreciate some background: how did you happen to notice this? Very few people have shown interest in the source details. Is there a specific feature/application that you are working on where this came up or just for overall consistency?

PS.

"values": [
        {
          "path": "sources.sensors.1W.0316013faeff",
          "value": 301.15
        }
      ]

is totally incorrect: source is metadata about a value update in the delta, and definitely should not be in the values array. However node server does currently no validation on the input and will happily digest this.

[sailoog]

@tkurki thanks a lot for your explanation. Now some behaviours make sense.

Some background:

We are fully implementing Signal K in the next OpenPlotter version. One will be able to set digital and analog sensors easily through a GUI (I2C, 1W, SPI, GPIO sensors...) and to select a Signal K path for every sensor to publish values on. A real example (really long sorry):

{
  "vessels": {
    "urn:mrn:imo:mmsi:000000000": {
      "uuid": "urn:mrn:imo:mmsi:000000000",
      "mmsi": "000000000",
      "navigation": {
        "position": {
          "source": {
            "type": "NMEA0183",
            "sentence": "GLL",
            "label": "signalk/signalk-parser-nmea0183",
            "talker": "GP"
          },
          "timestamp": "2016-07-29T16:57:03.000Z",
          "longitude": x.xxxxxxxxxxxxxxx,
          "latitude": xx.xxxxxxxxxxxxxx,
          "$source": "kplexOutput.GP",
          "sentence": "GLL"
        },
        "courseOverGroundTrue": {
          "value": 0,
          "$source": "kplexOutput.GP",
          "timestamp": "2016-07-29T16:57:02.000Z",
          "sentence": "VTG"
        },
        "speedOverGround": {
          "value": 0.26802562345538017,
          "$source": "kplexOutput.GP",
          "timestamp": "2016-07-29T16:57:02.000Z",
          "sentence": "VTG"
        },
        "magneticVariation": {
          "value": 0,
          "$source": "kplexOutput.GP",
          "timestamp": "2016-08-29T16:57:03.000Z",
          "sentence": "RMC"
        },
        "courseOverGroundMagnetic": {
          "value": 0,
          "$source": "kplexOutput.GP",
          "timestamp": "2016-07-29T16:57:02.000Z",
          "sentence": "VTG"
        },
        "gnss": {
          "source": {
            "type": "NMEA0183",
            "sentence": "GGA",
            "label": "signalk/signalk-parser-nmea0183",
            "talker": "GP"
          },
          "timestamp": "2016-07-29T16:57:02.968Z",
          "quality": 2,
          "satellites": 9,
          "antennaAltitude": 31,
          "horizontalDilution": 1,
          "geoidalSeparation": 49,
          "differentialAge": 0,
          "differentialReference": 0,
          "$source": "kplexOutput.XX"
        },
        "headingMagnetic": {
          "value": 6.1714844048,
          "$source": "I2Csensors.MPU-9250",
          "timestamp": "2016-07-29T16:57:02.829Z"
        },
        "attitude": {
          "roll": {
            "value": 2.22103411108,
            "$source": "I2Csensors.MPU-9250",
            "timestamp": "2016-07-29T16:57:02.829Z"
          },
          "pitch": {
            "value": 1.54085246986,
            "$source": "I2Csensors.MPU-9250",
            "timestamp": "2016-07-29T16:57:02.829Z"
          }
        }
      },
      "notifications": {
        "keyword": {
          "opencpn": {
            "message": "null",
            "$source": "notifications.opencpn",
            "timestamp": "2016-07-29T16:54:33.042Z"
          },
          "sms": {
            "message": "null",
            "$source": "notifications.sms",
            "timestamp": "2016-07-29T16:54:33.042Z"
          },
          "undefined": {
            "message": "null",
            "$source": "notifications.undefined",
            "timestamp": "2016-07-29T16:54:33.042Z"
          }
        },
        "mqtt": {
          "topic": {
            "message": null,
            "$source": "notifications.mqtt",
            "timestamp": "2016-07-29T16:54:33.108Z",
            "subtopic": {
              "message": null,
              "$source": "notifications.mqtt",
              "timestamp": "2016-07-29T16:54:33.108Z"
            }
          }
        },
        "gpio": {
          "input": {
            "gpio22": {
              "message": "L",
              "$source": "notifications.gpio22",
              "timestamp": "2016-07-29T16:54:33.224Z"
            }
          },
          "output": {
            "gpio25": {
              "message": "L",
              "$source": "notifications.gpio25",
              "timestamp": "2016-07-29T16:54:33.224Z"
            }
          }
        }
      },
      "propulsion": {
        "engine1": {
          "temperature": {
            "value": 302.15,
            "$source": "1Wsensors.0316013faeff",
            "timestamp": "2016-07-29T16:57:02.515Z"
          }
        },
        "engine2": {
          "temperature": {
            "value": 302.025,
            "$source": "1Wsensors.0415b2192bff",
            "timestamp": "2016-07-29T16:57:01.585Z"
          }
        }
      },
      "environment": {
        "outside": {
          "pressure": {
            "value": 101194.000244,
            "$source": "I2Csensors.BMP180",
            "timestamp": "2016-07-29T16:57:02.874Z"
          },
          "temperature": {
            "value": 302.45,
            "$source": "I2Csensors.BMP180",
            "timestamp": "2016-07-29T16:57:02.874Z"
          }
        },
        "inside": {
          "humidity": {
            "value": 53.9910850525,
            "$source": "I2Csensors.HTU21D",
            "timestamp": "2016-07-29T16:57:02.875Z"
          },
          "temperature": {
            "value": 302.88,
            "$source": "I2Csensors.HTU21D",
            "timestamp": "2016-07-29T16:57:02.875Z"
          }
        }
      }
    }
  },
  "version": "0.1.0",
  "sources": {
    "kplexOutput": {
      "label": "kplexOutput",
      "timestamp": "2016-07-29T16:57:02.968Z",
      "GP": {
        "talker": "GP",
        "sentences": {
          "RMC": "2016-08-29T16:57:03.000Z",
          "VTG": "2016-07-29T16:57:02.000Z",
          "GLL": "2016-07-29T16:57:03.000Z"
        }
      }
    },
    "notifications": {
      "label": "notifications",
      "type": "notifications",
      "opencpn": {
        "n2k": {
          "src": "opencpn",
          "pgns": {
            "undefined": "2016-07-29T16:54:33.042Z"
          }
        }
      },
      "sms": {
        "n2k": {
          "src": "sms",
          "pgns": {
            "undefined": "2016-07-29T16:54:33.042Z"
          }
        }
      },
      "undefined": {
        "n2k": {
          "src": "undefined",
          "pgns": {
            "undefined": "2016-07-29T16:54:33.042Z"
          }
        }
      },
      "mqtt": {
        "n2k": {
          "src": "mqtt",
          "pgns": {
            "undefined": "2016-07-29T16:54:33.108Z"
          }
        }
      },
      "gpio22": {
        "n2k": {
          "src": "gpio22",
          "pgns": {
            "undefined": "2016-07-29T16:54:33.224Z"
          }
        }
      },
      "gpio25": {
        "n2k": {
          "src": "gpio25",
          "pgns": {
            "undefined": "2016-07-29T16:54:33.224Z"
          }
        }
      }
    },
    "1Wsensors": {
      "label": "1Wsensors",
      "type": "1W",
      "0316013faeff": {
        "n2k": {
          "src": "0316013faeff",
          "pgns": {
            "undefined": "2016-07-29T16:57:02.515Z"
          }
        }
      },
      "0415b2192bff": {
        "n2k": {
          "src": "0415b2192bff",
          "pgns": {
            "undefined": "2016-07-29T16:57:01.585Z"
          }
        }
      }
    },
    "I2Csensors": {
      "label": "I2Csensors",
      "type": "I2C",
      "MPU-9250": {
        "n2k": {
          "src": "MPU-9250",
          "pgns": {
            "undefined": "2016-07-29T16:57:02.829Z"
          }
        }
      },
      "BMP180": {
        "n2k": {
          "src": "BMP180",
          "pgns": {
            "undefined": "2016-07-29T16:57:02.874Z"
          }
        }
      },
      "HTU21D": {
        "n2k": {
          "src": "HTU21D",
          "pgns": {
            "undefined": "2016-07-29T16:57:02.875Z"
          }
        }
      }
    }
  }
}

As you can see we are doing an unorthodox use of "notifications" group but it works really well. We store SMS and MQTT messages, opencpn alarms messages and even we store GPIO states.

Ignore the sources subtree. Is treating sensors id as N2K data but as I said, sending "Type" and "src" properties in delta messages was the only way that we could get something coherent on $source "reference" in the full tree.

Why do we need the $source info?

1. We have built a Signal K inspector to see what is happening in the system and sources is a very useful info about values to know and find out if you have more than one source for the same value , etc.
2. This is the most important reason. We have a trigger/actions system which is able to use any value from the signal k database as trigger of multiple actions to interact with the whole system. Besides values, we want also to be able to use timestamp and $source as triggers and evaluate their contents. This way, the potential interaction increases a lot.

About your suggestion:

$source:"<bus id>.<sensor id>"
$source":"1W.0316013faeff"
$source":"I2C-0.BMP180"
$source":"I2C-1.HTU21D"

That would be great. We don't need the "sensors" level at all, it's in my suggestion because now the provider id always appears as top level and it would be the unique provider for sensors. As you can see in the long example, now the only way of having different paths for $source is creating a provider by type of sensor and we don't want this (too much udp connections).

So if I understand correctly at this moment there is no way of achieve the path you suggest or can I achieve $source:"<bus id>.<sensor id>" somehow? Is there plans of implementing support for non NMEA sources on v1?

P.S

"values": [
        {
          "path": "sources.sensors.1W.0316013faeff",
          "value": 301.15
        }
      ]

The specification wiki is full of this kind of examples: https://github.com/SignalK/specification/wiki/Samples---NMEA-0183-RMC even in the multiple values handling solution.

[tkurki]

Side note:

UDP is connectionless, so I don't see how having lots of udp providers, as each update over udp is just one packet afaik.

However you might run against UDP packet max length with long deltas.

[tkurki]

I definitely think that adding support for non-NMEA data sources is something we should do. As long as we don't change the existing source structure for NMEA data we are completely backwards compatible.

[tkurki]

sigh the wiki is not in a very consistent shape and nor are our different server implementations.

[tkurki]

I think the best course of action would be to start with the deltas that produce the full tree you posted above. Do you have those handy? I could then create a test that starts the server with a matching configuration, inserts those and asserts that the resulting full tree has the correct $sources and /sources.

BTW my comment earlier about the server not caring about configured provider ids was wrong. The label in the delta is overwritten by provider id - as you've surely noticed. This seemed like a good idea at the time, but can be changed.

[sailoog]

You are right about UDP connections but we were concerned about raspberry performance. We try to make deltas as short as possible to not exceed the limit.

Agree. Since improving and expanding the NMEA features is the leitmotiv of signal k, I think supporting non NMEA sources and make easy dealing with custom sensors will make the difference.

I think we could go with this way or disable sources features provisionally in openplotter until signalk-node-server can manage non NMEA sources.

This is the current settings file but new UDP providers should be added:

{
    "vessel" : {
        "mmsi" : "000000000",
        "uuid" : "urn:mrn:imo:mmsi:000000000"
    },
    "pipedProviders" : [{
            "pipeElements" : [{
                    "type" : "providers/tcp",
                    "options" : {
                        "host" : "127.0.0.1",
                        "port" : "10110"
                    }
                }, {
                    "type" : "providers/nmea0183-signalk",
                    "optionMappings" : [{
                            "toOption" : "selfId",
                            "fromAppProperty" : "selfId"
                        }, {
                            "toOption" : "selfType",
                            "fromAppProperty" : "selfType"
                        }
                    ]
                }
            ],
            "id" : "kplexOutput"
        }, {
            "pipeElements" : [{
                    "type" : "providers/udp",
                    "options" : {
                        "port" : "55556"
                    }
                }, {
                    "type" : "providers/liner"
                }, {
                    "type" : "providers/from_json"
                }
            ],
            "id" : "I2Csensors"
        }, {
            "pipeElements" : [{
                    "type" : "providers/udp",
                    "options" : {
                        "port" : "55557"
                    }
                }, {
                    "type" : "providers/liner"
                }, {
                    "type" : "providers/from_json"
                }
            ],
            "id" : "1Wsensors"
        }, {
            "pipeElements" : [{
                    "type" : "providers/udp",
                    "options" : {
                        "port" : "55558"
                    }
                }, {
                    "type" : "providers/liner"
                }, {
                    "type" : "providers/from_json"
                }
            ],
            "id" : "notifications"
        }, {
            "pipeElements" : [{
                    "type" : "providers/execute",
                    "options" : {
                        "command" : "actisense-serial /dev/ttyOP_N2K"
                    }
                }, {
                    "type" : "providers/liner"
                }, {
                    "type" : "providers/n2kAnalyzer"
                }, {
                    "type" : "providers/n2k-signalk"
                }
            ],
            "id" : "CAN-USB"
        }
    ],
    "interfaces" : {}

}

And this would be the delta for 1W (for I2C type:I2C, src: BMP180...):

{
  "context": "vessels.urn:mrn:imo:mmsi:000000000",
  "updates": [
    {
      "source": {
        "type": "1W",
        "src": "0316013faeff"
      },
      "timestamp": "2016-07-28T18:18:46.074Z",
      "values": [
        {
          "path": "propulsion.engine1.temperature",
          "value": 301.837
        }
      ]
    }
  ]
}

[tkurki]

Continue the discussion at https://github.com/SignalK/specification/pull/238 since this is a spec issue first and node server issue second.

[sailoog]

Just to know, no pressing. I see that this has been merged to specification. Do you have plans to implement this on server-node soon?

[tkurki]

@sailoog sorry about leaving you on hold for so long. The PR I linked above adds support for generating arbitrary hierarchies under /sources based on the value in $source value.

Example: a delta with "$source": "i2c-0.0x48.amps" will generate

  "i2c-0": {
    "0x48": {
      "amps": {}
    }
  }

under /sources.

Working on this it occurred to me that adding a timestamp as a leaf property might be useful.

How does this suit the way you use / would want to use /sources and $source?

[sailoog]

Do not worry, really busy here too. I am quite sure it will suit. Thanks.

[sailoog]

Hi @tkurki, I am trying to use "$source" in deltas and getting an error so I guess this is not implemented yet. I am about to publish an important release of openplotter, do you have plans to implement this soon? thanks.

[tkurki]

It should I think. What are you feeding in and what is the error you are seeing?

[tkurki]

See https://github.com/SignalK/specification/blob/master/test/sources.js#L38-L52 on what is expected to work.

[sailoog]

I am trying to pass this by UDP 127.0.0.1:55557

{
  "updates": [
    {
      "$source": "1W.0316013faeff",
      "values": [
        {
          "path": "propulsion.engine1.temperature",
          "value": 301.837
        }
      ]
    }
  ]
}

But also fails with:

{
  "context": "vessels.urn:mrn:imo:mmsi:200000000",
  "updates": [
    {
      "$source": "1W.0316013faeff",
      "values": [
        {
          "path": "propulsion.engine1.temperature",
          "value": 301.837
        }
      ]
    }
  ]
}

This is the error:

TypeError: Cannot set property 'label' of undefined at /home/pi/.config/signalk-server-node/lib/pipedproviders.js:47:31 at Array.forEach (native) at FromJson. (/home/pi/.config/signalk-server-node/lib/pipedproviders.js:46:22) at emitOne (events.js:96:13) at FromJson.emit (events.js:188:7) at readableAddChunk (_stream_readable.js:176:18) at FromJson.Readable.push (_stream_readable.js:134:10) at FromJson.Transform.push (_stream_transform.js:128:32) at FromJson._transform (/home/pi/.config/signalk-server-node/providers/from_json.js:35:10) at FromJson.Transform._read (_stream_transform.js:167:10) at FromJson.Transform._write (_stream_transform.js:155:12)

This is my settings file:

{
  "vessel": {
    "uuid": "urn:mrn:imo:mmsi:98765432"
  },
  "pipedProviders": [
    {
      "pipeElements": [
        {
          "type": "providers/udp",
          "options": {
            "port": "55556"
          }
        },
        {
          "type": "providers/nmea0183-signalk",
          "optionMappings": [
            {
              "toOption": "selfId",
              "fromAppProperty": "selfId"
            },
            {
              "toOption": "selfType",
              "fromAppProperty": "selfType"
            }
          ]
        }
      ],
      "id": "kplexOutput"
    },
    {
      "pipeElements": [
        {
          "type": "providers/udp",
          "options": {
            "port": "55557"
          }
        },
        {
          "type": "providers/liner"
        },
        {
          "type": "providers/from_json"
        }
      ],
      "id": "OP_sensors"
    },
    {
      "pipeElements": [
        {
          "type": "providers/udp",
          "options": {
            "port": "55558"
          }
        },
        {
          "type": "providers/liner"
        },
        {
          "type": "providers/from_json"
        }
      ],
      "id": "notifications"
    },
    {
      "pipeElements": [
        {
          "type": "providers/udp",
          "options": {
            "port": "55559"
          }
        },
        {
          "type": "providers/liner"
        },
        {
          "type": "providers/from_json"
        }
      ],
      "id": "serial"
    },
    {
      "pipeElements": [
        {
          "type": "providers/execute",
          "options": {
            "command": "cat /dev/ttyOP_SKVA"
          }
        },
        {
          "type": "providers/liner"
        },
        {
          "type": "providers/from_json"
        }
      ],
      "id": "serial"
    },
    {
      "pipeElements": [
        {
          "type": "providers/udp",
          "options": {
            "port": "55561"
          }
        },
        {
          "type": "providers/liner"
        },
        {
          "type": "providers/from_json"
        }
      ],
      "id": "wifi"
    },
    {
      "pipeElements": [
        {
          "type": "providers/execute",
          "options": {
            "command": "actisense-serial /dev/ttyOP_N2K"
          }
        },
        {
          "type": "providers/liner"
        },
        {
          "type": "providers/n2kAnalyzer"
        },
        {
          "type": "providers/n2k-signalk"
        }
      ],
      "id": "CAN-USB"
    }
  ],
  "interfaces": {

  }
}

[tkurki]

Gotcha! Definitely an error. Will take a few days to find the time on compter to fix. Simplest would be to add a check here

https://github.com/SignalK/signalk-server-node/blob/master/lib/pipedproviders.js#L49

that source exists and only set label then.

[sailoog]

Glad to hear that, I was getting crazy... Unfortunately my js skill is not enough to help here. Let me know when you fix this please, thanks!

[tkurki]

The immediate problem should be fixed now and there are no errors, but the $source handling is not complete and you end up with "no_source" sources in the full model. Need to fix those separately.

What is your target for releasing the next version?

[sailoog]

Now I am getting:

ERROR:websocket:error from callback <function <lambda> at 0x755745b0>: 'source'

Annoying but since is not an stopping error I think I can go with this. It will be an alpha version so the target is no fatal errors. The main goal of this release is to become a self-updating image. OpenPlotter code, dependencies (SK included), opencpn, plugins and even raspbian will be able to be updated from OP interface. Hopefully, this will be the last image users will have to burn. Thanks.

[tkurki]

Is there anything more to the error message? Where do you see it? It does not look like a Node error message.

[sailoog]

Anything else. Every time I send a delta message by UDP I get the error. I see this on the terminal after starting sk node server and delta messages are sent. They stop when delta messages stop.

ERROR:websocket:error from callback <function <lambda> at 0x755a95b0>: 'source'
ERROR:websocket:error from callback <function <lambda> at 0x755a95b0>: 'source'
ERROR:websocket:error from callback <function <lambda> at 0x755a95b0>: 'source'
ERROR:websocket:error from callback <function <lambda> at 0x755a95b0>: 'source'
ERROR:websocket:error from callback <function <lambda> at 0x755a95b0>: 'source'
ERROR:websocket:error from callback <function <lambda> at 0x755a95b0>: 'source'
ERROR:websocket:error from callback <function <lambda> at 0x755a95b0>: 'source'

[dmdelorme]

I am also getting this

GET /signalk/v1/api/vessels/self 200 35.905 ms - 19 /home/pi/.config/signalk-server-node/lib/pipedproviders.js:47 update.source.label = providerConfig.id; ^

TypeError: Cannot set property 'label' of undefined at /home/pi/.config/signalk-server-node/lib/pipedproviders.js:47:31 at Array.forEach (native) at FromJson. (/home/pi/.config/signalk-server-node/lib/pipedproviders.js:46:22) at emitOne (events.js:96:13) at FromJson.emit (events.js:188:7) at readableAddChunk (_stream_readable.js:176:18) at FromJson.Readable.push (_stream_readable.js:134:10) at FromJson.Transform.push (_stream_transform.js:128:32) at FromJson._transform (/home/pi/.config/signalk-server-node/providers/from_json.js:35:10) at FromJson.Transform._read (_stream_transform.js:167:10) [Errno 111] Connection

refused

should i update signalK

[tkurki]

@dmdelorme That one I fixed 3 days ago. You can update and get the xi simply with git pull.

[tkurki]

@sailoog I am unable to reproduce your problem.

I have server started with bin/signalk-server -s settings/openplotter.json (openplotter.json is the one you sent) and I have one websocket client open.

Then I am sending a delta from the command line:

echo -n '{"context": "vessels.urn:mrn:imo:mmsi:000000000","updates": [{"source": {"type": "1W","src": "0316013faeff"},"timestamp": "2016-07-28T18:18:46.074Z","values": [{"path": "propulsion.engine1.temperature","value": 301.837}]}]}
' | nc -4u localhost 55556

No errors on the server console, delta shows up in the client.

[tkurki]

BTW you don't necessarily need liner if you do not send the newline in the UDP message: UDP sends one datagram at a time, which in your case means one delta.

[sailoog]

@tkurki I have an idea about where the error could come, let me try something. Probably you are right and this is not a Node error, it could be an error generated by the openplotter actions system trying to parse delta messages with old source format.

[sailoog]

Fixed websocket error. Waiting now for complete $source implementation, thanks @tkurki.

[dmdelorme]

@tkurki i think you are doing a great job love the concept, sailoog latest fixes seem to have corrected most of my issues.. My goal for the system is 2 fold 1 navigation 2 mapping ie collecting and processing data. I live in Canada sorry eh. all the maps are only available in BSBV4 format but with good data. SK outputed data even though Openplotter was having growing pains. The combination of SK, OpenPlotter and OpenCPN, Kplex is going to change the game with a little sweat.

[tkurki]

@sailoog I think I have it fixed now. Could you please install the master version of signalk-schema with npm install signalk/specification and try it out.

Before the upgrade you should be seeing $source: "no_source" data in the full tree.

When the fix is working those should be replaced with the real $source data in the incoming delta.

You can verify that you are running the fixed version by checking for example for this line https://github.com/SignalK/specification/blob/master/test/fullsignalk.js#L160 in node_modules/signalk-schema/test/fullsignalk.js

Let me know how this works for you - I can then release a new version of the signalk-schema package (from the signalk/specification github repo) and upgrade the server's package.json to use it.

[tkurki]

@dmdelorme I am curious: how do you store the data you get and how do you process it? (might want to continue this on SignalK Slack if you're interested).

[sailoog]

it works like a charm! thanks @tkurki

[dmdelorme]

the data is all live and i have been using signalK as the central hub. Inputs and outputs though openplotter, node-red, kplex. Node-red is easy but a real system sucker on a pi but it is getting better. I could store the data but. At this point the quality of the data and reliability are more important as its main propose is to be primary navigation so the system should be resilient and fault tolerant, hardware and software. Secondary mapping and storing data Psql opensea map etc. For viewing maps/charts I use OpenCpn on 2 pi 3 via ssh or vnc via phone,linux laptop, etc. Even though there has been a few hicups in the short term the fact is i still had primary data flowing though SK. I interpreted that as resilient. SignalK Slack??? I am old and Canadian .. goggle found invite... map ???