Definition of "what is a value"

[mxtommy]

As I've started trying to parse the full tree for my signalk app, I've come across the problem of trying to figure out what signalk paths are available on the server. It seems there's no easy automatable way to do this from the consumer.

My idea (like I'm sure a lot of other UI projects have) is to dynamically generate the widgets/graphs/etc based on what the server has. Currently I see two choices of finding out.

A) having the json schema bundled with the app. I could then compare the paths available on the server to the spec and know what it is (hopefully). Big problem with this is whenever something is updated in the spec you need to update the consumer as well or it wont know how to handle it. In the real world this is not good I think :)

B) Recursively walk the full json tree I get back from the server, and try and make "guesses" about what is what. This is full of corner cases and is error prone. Currently I'm making my guess on what objects in the json tree are values by the presence of a "timestamp" property. But then, you have to further guess as to what type of data is it? Is it a number value under the "value" property? Or is it one of the many "complex" objects where the data is under different keys? (lattitude/longitude/setTrue/etc). While you can start coding some logic in there to deal with each corner case, that does not scale and new corner cases will come up all the time...

This all makes writing a consumer of signalk data extensively more complicated than I feel it could be.

I propose that signalK needs a definition of what constitutes a "value" object.

I think the easiest way is to eliminate both the "simple" property leaf. (like vessels.self.name) and "complex" ones (like navigation.position, where lat/lon both are specified).

Standardize on and object with $source, timestamp, value properties. There's no harm in vessels.self.name having a source/timestamp, but it having it makes it clear it's a "value".

There seems to be some existing discussion in PR#315

I'd also suggest having another property: "dataType". It would be an enum and would let you know what to expect the format of .value to be. Such as "number", "string", "lat/lon/altitude","GEOJson", "timestamp", another object etc. I know technically it seems to be stored in the spec under the objects type, but when you retrieve the full model from the server it is not returned. Unless you already have the spec (see point "A" above) you have no way to know what type of data it is. This would be SOOO helpful I think in UI's so you could know which signalk paths would work with which widgets automatically, without having to hard code something that will likely change often (the spec).

Just for fun, if we're doing all that, I suggest a new optional property, "time-to-live". Combined with the timestamp would be possible for UI's to show that data is expired/old (or not) (either by specing that the server send a null value delta to consumers, or leaving it up to consumers to implement it.)

I'd be willing to do a bit of the grunt work if needed. That said, I have "zero" experience with tests/testing, so would need someone to help with that aspect of it.

[sbender9]

Note that timeout is there in meta : https://github.com/SignalK/specification/blob/c3d072f42893b56966ad43f2fcfc726dae995939/schemas/definitions.json#L548-L553

[mxtommy]

Ah, good to know regarding timeout! Still learning the spec and don't know everything yet :)

[joabakk]

Although we are still pre v1 of the spec, it's a bit late in the game to make such fundamental changes. A lot of effort and integrations rely on the current structure.

[mxtommy]

I guess adding a "dataType" might be something post V1 if it's needed at all.

That said, being able to point to a signalK path and know without 15 corner cases how to parse it should be simple and fundamental I think.

Take for example navigation.speedOverGround. I know I can append .value to that and get the current value. I can append .meta and get the metadata, .timestamp for the timestamp, etc.. Most signalk paths seem to already follow this pattern.

Now take navigation.attitude.roll. If I want the value, well, that's the value. If I want the metadata/timestamp, I have to go up a level all of a sudden? It's not consistent with most of the others that have .value. Same thing for vessel.self.name. It's a string, not even metadata etc. When trying to parse automatically this makes a bunch of corner cases.

I also don't think it would be that big of a change. A large percentage of objects already have $source/timestamp/value. It would be bringing the other paths to be like that.

I think there's also a backlog item in the V1 project related to this (didn't see it before) "Replace object structured data points with individual data points".

[tkurki]

Post version 1 I would like to move towards multiple, granular APIs rather than a single, all encompassing model. One such api would be "hey, what are all the data points that you've seen for self" and the result could be data structure that you can simply iterate, not parse using some heuristics.

[tkurki]

315 may bring some clarity to this issue.