Open universal json format

[tkurki]

Navgauge uses its own 'universal' format for sending basic navigation data to the browser. 'universal' and 'basic' mean that data from different sources (NMEA 0183, N2K, different sources within a N2K network) are combined on the server and sent in the same format and same fields so that the UI doesn't need to know anything about where the data is coming from. The combining is done in a boat specific configuration file in Navgauge.

This 'universal' way is something I just came up with when making the UI work universally with the data sets from three different boats that I had available. For example units are a mixture of metric & marine unit.

Creating a more universal and standard format would support sharing innovation & functionality between different efforts.

[rob42]

@tkurki - bacon is nice in js, but we cant really force the industry to js everywhere :-) So my use case stands - a common data model with easy replication?

On another note I have some (very) basic code running in the freeboard server - it translates my 3char format into the universal model. I have merge functionality working and can read my instruments. Its easy to send the whole thing out over websockets, but i will try to get it to send only changes. When it reads AIS data it creates another vessel in the top-level array and updates that. The static data is saved to file and loaded again at reboot- possibly it should be in meta-data A sample (truncated a bit)

[{ "self": {
            "name": "motu",
            "mmsi": "2345678", "source": "self", "timezone": "NZDT",
            "navigation": {
                "headingMagnetic": {"value": 114, "timestamp": "2014-03-29T06:49:16.135Z", "source": "self" },
                "declination": {"value": 0, "timestamp": "2014-03-29T06:49:16.136Z", "source": "self" },
                "rateOfTurn": {"value": "118.26", "timestamp": "2014-03-29T06:49:16.126Z", "source": "self" },
                "pitch": {"value": "4.97", "timestamp": "2014-03-29T06:49:16.130Z", "source": "self" } 
                },
            "communication": {
               ... },
            "alarms": {
                "silentInterval": {"value": 600, "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
                "windAlarmMethod": {"value": "sound", "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
                "windAlarmState": {"value": "off", "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
              ... },
            "power": {
                "normalVoltage": {"value": 12.9, "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
               ... },
            "propulsion": [
                {
                    "outboard": {
                        "type": {"value": "petrol", "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
                        "state": {"value": "started", "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
                        "rpmAlarm": {"value": 5000, "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
                    ... } 
                    }],
            "tanks": [{... } ],
            "steering": {
                "autopilot": {
                    "state": {"value": "on", "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
                    "mode": {"value": "powersave", "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
                    "targetHeadingNorth": {"value": 23, "source": "self", "timestamp": "2014-03-24T00:15:41Z" },
                  ... } 
                  }  
          }}]

[tkurki]

+1 on the common data model.

Just sharing the advantages of FRP in this problem domain.

lauantai 29. maaliskuuta 2014 rob42 notifications@github.com kirjoitti:

@tkurki https://github.com/tkurki - bacon is nice in js, but we cant really force the industry to js everywhere :-) So my use case stands - a common data model with easy replication?

On another note I have some (very) basic code running in the freeboard server - it translates my 3char format into the universal model. I have merge functionality working and can read my instruments. Its easy to send the whole thing out over websockets, but i will try to get it to send only changes. When it reads AIS data it creates another vessel in the top-level array and updates that. The static data is saved to file and loaded again at reboot- possibly it should be in meta-data A sample (truncated a bit)

[{ "self": { "name": "motu", "mmsi": "2345678", "source": "self", "timezone": "NZDT", "navigation": { "headingMagnetic": {"value": 114, "timestamp": "2014-03-29T06:49:16.135Z", "source": "self" }, "declination": {"value": 0, "timestamp": "2014-03-29T06:49:16.136Z", "source": "self" }, "rateOfTurn": {"value": "118.26", "timestamp": "2014-03-29T06:49:16.126Z", "source": "self" }, "pitch": {"value": "4.97", "timestamp": "2014-03-29T06:49:16.130Z", "source": "self" } }, "communication": { ... }, "alarms": { "silentInterval": {"value": 600, "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, "windAlarmMethod": {"value": "sound", "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, "windAlarmState": {"value": "off", "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, ... }, "power": { "normalVoltage": {"value": 12.9, "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, ... }, "propulsion": [ { "outboard": { "type": {"value": "petrol", "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, "state": {"value": "started", "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, "rpmAlarm": {"value": 5000, "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, ... } }], "tanks": [{... } ], "steering": { "autopilot": { "state": {"value": "on", "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, "mode": {"value": "powersave", "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, "targetHeadingNorth": {"value": 23, "source": "self", "timestamp": "2014-03-24T00:15:41Z" }, ... } } }}]

Reply to this email directly or view it on GitHubhttps://github.com/tkurki/navgauge/issues/4#issuecomment-38988861 .

[rob42]

@tkurki - thanks - just to be clear the common data model should not be mandatory internally, its just a model to give every current and future datapoint a unique address in a common structure, kind of like the internet really. If bacon does it better internally, then go for it - eg I am still using my 3char code to the arduinos cos i dont have RAM or CPU spare for json.

But externally we should all present/expect the same model. Does any-one have comments or changes on the universal model I put up? (I updated it)

Should routes,charts,waypoints be inside 'motu'? or is 'source' enough?

So we should also define the REST interface. I like the /meta, /data, /byName, /byMmsi, etc. But it needs a spec. as to urls and what they return

And how to name vessels in a universally unique way?

[rob42]

What is a good std label for your own boat in the vessels array? I used 'self' but that potentially confuses js code since 'self' is synonymous with 'window', at least sometimes...

[timmathews]

@rob42 NMEA 0183 has a sentence called "Own Ship Data", perhaps instead of self, you could use ownShip. Or localboat (like localhost)?

[tkurki]

I added location [lon,lat] yesterday I think. localBoat - I like that.

Come to think of it resources are identified by uris and in this context the 'primary key' would be the uri where you get the model for the vessel. Name, imsi, call sign etc are really metadata around that. Vessels api call would give you all the vessels this server knows about. There can be duplicate names, some may not have imsi or radio equipment at all, but in this context all should have a single model uri. What do you think?

[rob42]

@tkurki - localBoat - I like that too. Some sort of unique identifier scheme is needed for three use cases I have to deal with

• AIS - lots of ships sending updates - you have to match with their data record - they all have imsi
• There are 9 pleasure boats called 'Infinity' in NZ!
• Dual engines - how to map NMEA to the engine?

Maybe we use imsi>vessel name+callsign>vessel name>vesselName(n) in that order of preference?

[tkurki]

Should headingNorth be headingTrue?

[rob42]

Or heading: { north, magnetic} I used originally  north because true is a reserved word It became headingNorth m deterrence to TimMattews aversion to extra keystrokes i : -) Rob

-------- Original message --------

From: tkurki notifications@github.com

Date:31/03/2014 03:15 (GMT+12:00)

To: tkurki/navgauge navgauge@noreply.github.com

Cc: rob42 robert@42.co.nz

Subject: Re: [navgauge] Open universal json format (#4)

Should headingNorth be headingTrue? — Reply to this email directly or view it on GitHub.

[timmathews]

@rob42 We could just generate a UUID at installation and use that. There are good UUID libraries for every platform, even Arduino. And while it may not be necessary for what we're doing, RFC 4122 defines a UUID URN namespace.

I think headingTrue and headingMagnetic would be the right way to go. Here is my model for the data that is generated by my GPS receiver (Garmin GPS17x). I have elected to use cogTrue and cogMagnetic for the course values generated.

The comments take the form of PGN:Field..+RepeatCount(Bits)

{
  "navigation" : {
    "gnssType"       : enum,            // 129029:6
    "method"         : enum,            // 129029:7
    "integrity"      : enum,            // 129029:8
    "fixMode"        : enum,            // 129540:1
    "satelliteCount" : integer,         // 129029:10(8), 129540:3(8)
    "satellites" : [
        {
            "prn"            : integer, // 129540:4..+7(8)
            "elevation"      : degrees, // 129540:5..+7(16)
            "azimuth"        : degrees, // 129540:6..+7(16)
            "snr"            : dB,      // 129540:7..+7(16)
            "rangeResiduals" : integer, // 129540:8..+7(32)
            "status"         : enum,    // 129540:9..+7
        }
    ],
    "dilutionOfPrecision" : {
        "desiredMode" : enum,           // 129539:1
        "actualMode"  : enum,           // 129539:2
        "horizontal"  : float,          // 129029:11(16), 129539:4(16)
        "vertical"    : float,          //                129539:5(16)
        "probable"    : float,          // 129029:12(16)
        "time"        : float,          //                129539:6(16)
    },
    "geoidalSeparation"     : meters,   // 129029:13(16)
    "referenceStationCount" : integer,  // 129029:14(8)
    "referenceStations"     : [
        {
            "stationType"   : enum,     // 129029:15..+3
            "stationId"     : integer,  // 129029:16..+3
            "correctionAge" : seconds,  // 129029:17..+3(16)
        }
    ],
    "latitude"    : degrees,    // 129025:0(32),    129029:3(64)
    "longitude"   : degrees,    // 129025:1(32),    129029:4(64)
    "altitude"    : meters,     //                  129029:5
    "cogTrue"     : degrees,    //      129026:1+3(16)
    "cogMagnetic" : degrees,    //      129026:1+3(16)
    "sog"         : m/s,        //      129026:4(16)
    "time"        : timestamp   //      129029:1+2(16+32)
  }
}

Which brings me to my next point: SIDs. These PGNs nearly all include a SID which tells the receiver of the data that they all belong together. I think that when we're publishing updates (via Websocket or UDP or whatever), it would make sense for the server to wait until it has received all of the data in a particular SID group before sending it off.

[rob42]

@timmathews - ok - but both heading* and cog* (headingTrue = cogTrue +- leeway)

The GPS data is excellent - I will plug it in.

SIDS - yes we are not packet size constrained, so no need to do ugly message chunking. General rule for all messages from other systems is to collect chunked messages and send as a single json. I think the same happens for NMEA0183 when you have 2 engines. First packet says which engine, next packet is for that engine - how ugly is that!

[rob42]

@timmathews - I might modify the gps info into its own object to separate it out a bit, I'm wary of future name collisions.

{
  "navigation" : {
    gps:{
          "gnssType"       : enum,            // 129029:6
          "method"         : enum,            // 129029:7
           ...
         "referenceStations"     : [
              ...
           ]
     },
    "latitude"    : degrees,    // 129025:0(32),    129029:3(64)
    "longitude"   : degrees,    // 129025:1(32),    129029:4(64)
    "altitude"    : meters,     //                  129029:5
    "cogTrue"     : degrees,    //      129026:1+3(16)
    "cogMagnetic" : degrees,    //      129026:1+3(16)
    "sog"         : m/s,        //      129026:4(16)
    "time"        : timestamp   //      129029:1+2(16+32)
  }
}

[rob42]

@tkurki - oops overwrote your depth stuff - its back in

"depthBelowTransducer",
                    "depthBelowKeel",
                    "depthBelowWater",
                    "depth",
                    "waterLineToTransducer",
                    "transducerToKeel",

[timmathews]

I might modify the gps info into its own object to separate it out a bit, I'm wary of future name collisions.

@rob42 Makes sense, but should probably be the more generic gnss instead of gps.

Also, I should have been more specific when I referenced my cog* values above, I meant them as an example of naming, not as a replacement for heading. *Mea culpa.

[rob42]

@timmathews - gnss it is.

In general I like nounAddnoun structure because its nice in code completion, eg heading? >> select headingTrue/headingMagnetic. Easier to remember and find names than trueHeading/magHeading style

Same with yyyy-mm-dd, it sorts nicely.

We need cog* anyway so we have both now.

[keesverruijt]

Isn't COG north referenced by definition? In other words you'd need to calculate the local magnetic deviation back in, is there equipment that does that and/or is there a public definition of the WMM?

----- Reply message ----- From: "rob42" notifications@github.com To: "tkurki/navgauge" navgauge@noreply.github.com Cc: "canboat" kees@verruijt.net Subject: [navgauge] Open universal json format (#4) Date: Mon, Mar 31, 2014 03:06

@timmathews - gnss it is.

In general I like nounAddnoun structure because its nice in code completion, eg heading? >> select headingTrue/headingMagnetic. Easier to remember and find names than trueHeading/magHeading style

Same with yyyy-mm-dd, it sorts nicely.

We need cog* anyway so we have both now.

— Reply to this email directly or view it on GitHub.

[rob42]

Declination is calculated in freeboard from public resources - see https://github.com/rob42/freeboard-server/blob/master/src/main/java/nz/co/fortytwo/freeboard/server/util/Magfield.java so we can have both.

[rob42]

Sending differences instead of the full model proved to be trivial once I though about it the right way. I have a processing chain that merges the incoming NMEA/3char/JSON into the master model. Then various routines further decorate the master. Then I send it out but its hard to know what changed without complex stuff. The answer was to create an empty temp 'vessels[]' object and merge all the NMEA/3char/JSON into that. Merge the temp object into the master, then the decoraters run against master, but update the temp. Final step is to merge the temp object into the master again, but send just the temp object out to clients.

[timmathews]

I think it is time we began documenting the data model formally and for that we should do something similar to Twitter's approach to API documentation and host it with GitHub pages. However, we should create a GitHub organization for the project first so that documentation changes can be done with git push instead of pull requests. I'll do the gnss block on the wiki in the Twitter format, but I don't think editing a wiki through GitHub is a long-term viable solution.

Any thoughts on a name? The best I could come up with is Open Marine Data Exchange Format (OMDEF).

[rob42]

http://en.wikipedia.org/wiki/JSON#JSON_Schema - I shudder to think JSON could be the new XML, but this does seem to be a good way to add structure to the definitions.

Yes we need more than a wiki page now - I think we are at the point of a separate project

We should think carefully on the name - its got to be a good one :-)

[rob42]

I found a name, signal flag K (Kilo) "I wish to communicate with you."

[rob42]

Plus we can use the flag image as logo and a "Kilo enabled" type of sticker on out projects

[fabdrol]

Hi! @rob42 pointed me here. I'm very eager to contribute. I'm considerably late to the party, so I'll refrain from being too opinionated until I've read through this thread twice, and read/tried the different projects.

What I've read so far is this: the JSON specification (for the sake of this - way too long - comment I shall call it KILO after @rob42's suggestion, so it's clear what we're talking about) seems to be forming nicely, and there seems to be general consent regarding (web) sockets as the transport.

The progress so far has inspired my imagination: I think the combination lends itself perfectly for a provider/consumer pattern.

A Provider (can be anything; NMEA 0183, 2000, SeaTalk) runs on a server (e.g. Raspberry Pi) and transforms it's input data into KILO format. A Consumer can be anywhere (in the network, or on the server itself) and does the opposite: consumes the (combined) KILO stream and translate it into whatever. A Consumer could in that case be a browser client that displays gauges, a navigation app on an iPad or an utility on the server that translates the combined data back to one big fat NMEA feed for another device on board. Or, indeed, it could be a proxy server for Siri that makes an iPhone taped to a VHF radio annoy everyone else on the water ;).

Well, those are my initial thoughts. What do you guys think? Bare in mind that I haven't yet checked out your code, so I hope I'm not stepping on anybody's toes here!

Since I can't sleep, let's think this out a bit more. A complete boat network would consist of the following:

• A central web server configured with the ship's info and a list of attached Providers. The web server does only three tasks: it merges the KILO format from the attached providers, broadcasts the combined KILO feed onto the network via websockets and exposes a simple REST API that allows Consumers to query a ship's own info, a ship's capabilities (the list of attached Providers) and information about a single Provider (which was cached at start-up by the web server). Providers don't communicatie with each other.
• Providers: command-line utilities that print lines of KILO to stdout. At startup of the utility a (normal?) JSON object with information about the identity and the capabilities of the providers is printed on stdout. The web server caches this to make this info available on the REST API. Providers, being CLI utilities, makes them language agnostic (a web server implemented in node could easily manage each process using child_process.fork).
• Consumers are programs listening to the central web server's WS stream. What they do with the data subsequently is up to them. In order to make browser-based consumers easier without spinning up a special web server, I'd propose adding a simple static file server to the central web server (it does kinda screw up the decoupling I'm aiming for here, but given the limited resources it's probably a more practical choice - as it's less expensive on the server than running another web/file server). Consumers don't communicatie with each other.

Main downside of this pattern is, probably, that it's not very well suited to send data back up the chain (e.g @rob42's case of controlling the autopilot). An option would be to have a hybrid Consumer/Provider to handle these kind of cases.

What do you guys think? Again, if I'm stepping on somebody's toes let me know! I've only read this thread - once. And it's 01:53 AM here in Amsterdam ;).

Fabian

EDIT: I'm completely aware that my long, long comment is a bit off-topic.. but if this is going to go somewhere I don't think it should stop at standardising the protocol. This should evolve into a complete open-source (both hardware and software) alternative the different commercial implementations of NMEA, seatalk and whatnot - disrupting and freeing the nautical instrument industry..... Okay, I am starting to sound ridiculous. I shall stop now ;).

[timmathews]

https://github.com/tkurki/navgauge/wiki/Global-Navigation-Satellite-System-(gnss)-Object is an example of how I think we should document. It probably needs an example object at the bottom. I'd like to put a JSON example in the description column, but markdown doesn't do well with multiple lines in table cells and I don't feel like switching to something else just now.

[timmathews]

@rob42 My favorite signal flag is J-Juliet: I am on fire and have dangerous cargo on board, but K-Kilo is probably a better logo. :smiley:

@fabdrol Welcome! I don't believe that any toes have been stepped on. You've pretty much gotten to parity with what we have so far just by thinking, so I think that's good.

As far as controlling things, there's no reason that your Consumers can't also be Producers (essentially RPC over WebSockets). I hadn't put a lot of thought into controlling things yet, but it is certainly a requirement.

[keesverruijt]

@rob42 My favorite signal flag is J-Juliet: I am on fire and have dangerous cargo on board, but K-Kilo is probably a better logo.

One could also argue that the description of X is appropriate: Stop carrying out your intentions and watch for my signals, but “Xray” doesn’t really do it as a name. So I agree with Kilo as well. Short, to the point, and not used for anything else in IT that I know of. Apparently the prefix is so dominant that nobody else has dared name a product or idea “Kilo”.

There is one issue with it, there is already a github.com/kilo user, so we’ll have to settle for /kilo.

Shall we use github.com/canboat/kilo as the external go-to point?

@fabdrol Welcome! I don't believe that any toes have been stepped on. You've pretty much gotten to parity with what we have so far just by thinking, so I think that's good.

Welcome, and your physical location is fortuitous as well as you’re only one hour away from me (Harlingen.) As far as controlling things, there's no reason that your Consumers can't also be Producers (essentially RPC over WebSockets). I hadn't put a lot of thought into controlling things yet, but it is certainly a requirement.

Indeed. The parts of CANboat that aren’t public yet concern two other “sources”. I use a Maxim 1 Wire network to measure temperature, which is read only. I use a WAGO Linux PLC running my own program that communicates with HTTP (sort of) and REST and JSON, which is very much two way as it is able to monitor and control all onboard circuits. In fact this is the reason that I started the entire project, not the N2K side of it.

But there is so much more that we can share. Last night I thought it would also be nice if we would be able to come up with a javascript interface that encapsulates a gauge (browser side). Then we can share our gauges, or collaborate on building the absolute best-ever multi faceted super gauge.

We are pretty strung out on the server side in technologies, and haven’t yet discussed much whether and if so which JS libraries you use. As CANboat is the oldest, it uses JQuery as that was the best player in field at the time (2009).

— Kees

[tkurki]

+1 for Kilo, but I think it needs something else to be googlable. I'm terrible with names, no good suggestions here. KiloSpec is name of a company, KiloJSON (as in GeoJSON) brings up only two hits?

+1 for the separate org on Github. Kilo is also already taken on Github.

[tkurki]

@fabdrol IMHO your thoughts are exactly in line with what has been discussed. Also a bit of diversity is a good thing. If I knew all I'd just stick to my own thoughts.

Speaking of which: I changed apparent|trueWind to windApparent|True and speedOverWater to speedThroughWater (STW).

[tkurki]

@canboat: I've tried to move towards widgets-as-components direction. Take a look at for example windgraph.js: it assumes that d3.js and bacon.js are available, takes in a selector as init argument (specifies where in the DOM tree to put the graph) and expects data to come in as onData(msg). Apart from that it is self-contained, client side javascript. This is exactly where the KiloJSON comes into play: for it to be pluggable incoming data should be in a common format that the gauge can understand.

And I have a 10 DOF IMU on my desk connected to my Raspberry, so multiple sources and sensors is a definite reality.

I would love to have a proof of concept running on the net - for this to be viable we need at least 2 reference implementations that actually interoperate...anybody pair with me to set this up?

Isn't the autopilot control channel best served with a REST api: websockets for streaming updates, rest for one shot control or query calls.

[rob42]

The name - What about just 'K', or 'Signal-K', 'Signal K', 'SignalK'- that has no hits on google. The meaning is just so well aligned with our direction, makes a great slogan: "I wish to communicate with you."

In terms of the convergence of our

[fabdrol]

@timmathews Thanks! I agree, that would be the best solution. As long as the communication back (Consumer -> Producer) doesn't go through the central web server (that way, the goal decoupling is still archieved without and the Consumer doesn't really need to know about a boat's capabilities. They just do their thing with the data they receive).

Last night I wrote that Providers could be registered using a list in a configuration file. Come to think about it, and that is maybe not the best solution. We'd end up with a lot of configuration per boat, which would have to be altered whenever a Producer is added. Instead a standard way of registering would prob be better, but then how'd we handle process management.. Also, if it runs on the server a user would have to go into that anyway. Final thought on registering: something like NPM specifically for Kilo Producers and Kilo Consumers would be cool (and could be handled by an "install" page in settings!).. Not important for now, but would be a cool addition in the future.

@canboat How about org name /kilo-project/? Regarding an encapsulated way of handling gauges (or Consumers in the browser in general), Web Components/Polymer comes to mind. Not fully standardised yet, bit very promising and it provides the necessary encapsulation and more. That settles it, the first meet-up will be here! I'm sure Tim doesn't mind sailing to the Netherlands! I'm originally from Friesland too, btw.

I'll set up a project implementing what I talked about. Goal mostly is standardising the pattern and the interface (but also a stable implementation). If the pattern/interface is standard and agreed upon by most, the language of the implementation won't matter anymore. Once I have a simple server running, I'll try to get some sample Providers up and some basic Consumers. One issue for me is hardware. On board I only have a tiller pilot accepting SeaTalk, a GPS emitting NMEA 0183, and some instruments emitting NMEA 0183. Furthermore, this would move way too slow if I had to come to the boat in order to work on it. How did you guys solve this/what recommendations can you give me?

[fabdrol]

@tkurki As I'm starting from scratch, I think I can easily pair with you on this. How do you want to proceed?

As for your example, you assume that d3.js is there.. I'm the world of widgets that can be dangerous - especially if you're not the only one writing widgets (e.g multiple versions of jquery conflicting). Another solution to the one I mentioned (web components) would be to use something like Browserify or AMD to define dependencies including version. This doesn't take care of encapsulation, that would be the responsibility of the widget author (e.g calling $.noConflict within the widget's own encapsulated namespace).

[ktuukkan]

Been a bit absent and haven't read all messages with deep thought, but I like "Kilo" as well, very nice. Also very well put @fabdrol, I agree with almost everything. I thinki we should concentrate less on the consumer side because the common format is our beef. Clients can use the data as they wish, and of course based on your experience we can shape the model better support different kind of consumers, but I think we shouldn't design for any particular use case except "sharing what the vessel knows". My project is mainly a data producer, so I have quite little to comment on consumer side (i.e. not not much personal experience) and it already seems to be forming very well.

Anyways, here's some more general guidelines I've been thinking:

• "forget" conventions and practices of NMEA et.al. as we have totally clean table here
• field names with camelCase
• fixed values/enums with CamelCase
• be consistent with naming, units, symbols and everything. For every occurence of "x", "x" means "x".
• sticking with chosen units avoids confusion on client side (e.g. depth always in meters)
• no irrelevant symbols. For example, NMEA 0183 transmits a lot unnecessary characters for units and they are useless if the spec defines which units are used in each field.
• avoid using abbreviations unless it's absolutely clear by common knowledge
• avoid arrays with plain indexing, e.g. vessels[0], if it requires consumer to "know" position of each block in beforehand

What do you think? Should we document this kind of things as well?

This thread is getting pretty long.. IRC, Skype anyone? :)

[fabdrol]

@ktuukkan I agree, a Consumer could be whatever.. After all, there isn't a lot more they can do but to simply read the WS feed and do there thing.

[rob42]

@fabdrol - hmm - REST interface for replies from the browser could work...

Some background here - I originally started freeboard because there was no way to provide instruments without buying into the big names. Some open source small projects - but all alone - no common structure. Only OpenCPN had escape velocity - but no suitable sensors/integration/hardware/etc. So I started freeboard partially to create a common platform.

But I have to code front-end (javascript), backend(java), microcontrollers (c, arduino), setup Rpi's, and Ive designed boards to make the physical connections from boat stuff to arduinos - too much!

But I think we are there (common platform) now - we all have our interests and strengths, and a common platform to work too - so if you create a cool client, there is data to show - and if you decode locked formats - there is a client to benefit. Plus new-comers with limited interests or skillsets can find a place to fit in.

So I think the next step is to start a new shared project (Signal K?), and link our projects into it. Then I think we pick the areas we are keen on, and work towards, integrating all our stuff together. Meanwhile we spread the word about - and see if we can reach that elusive escape velocity of the big projects!

If you agree we need to vote on the name - its going to be important - has to be memorable, easy to say, spell, google friendly, code friendly, what else?

Kilo (taken on Github already, flag and all) 1) K 2) Signal K - gets my vote 3) k-project 4) ?

Rob

[tkurki]

@fabdrol Take a look at http://navgauge.herokuapp.com/gauges.html . Ideally you should be able to enter your own server's url in the box, press connect and the gauges would start showing data from your server. (the box is now a placeholder, no functionality)

Navgauge has the ability to play back raw NMEA captures, Canboat analyzer output and Navgauge's own json capture. Handling raw Canboat actisense output and adding support for several NMEA sources is pretty straightforward. Whatever you do I suggest getting some data captures from real sailing, captures from a boat sitting still get boring pretty quick ;-). If you don't mind Node you can just start with cloning Navgauge and playing around with it.

Browserify, AMD, Bower etc +1, haven't done it because it's been just me as the user and not sure what to pick. Web Components seems to be still in the future, but maybe you have better info on that.

[fabdrol]

@tkurki Node is my language of choice as well, so that is a good start. Here's what I'm thinking: I'll set up a first implementation of the decoupled web server in my idea. It'll take a list of Providers as it's input, multiplex this data and emit it on WS. I'll start writing a simple Provider that takes GPS input (easy starting point, as there are already projects for that written in node). I'll make the WS stream available cross-domain, so you can start playing with it in Navgauge. That way we'll actually have two projects that are different in implementation, but communicate over WS in the same, shared protocol - which should be good for testing.

More practical: I think we need to fork a good (NMEA) parser and start work on the implementation of our new protocol - evolving the parsers as the protocol evolves. My thinking is, if we have a few basic parsers available (SeaTalk, NMEA 0183 & 2000 to start with?) of which we make sure they follow the specification of the protocol with each new version, new contributors have a good place to get started - either as contributor to the parser or building something on top of a parser.

That said; maybe we should vote for a name quickly, set up the new organisation on github and set up a few basic things for collaboration (e.g. an IRC channel for communicating, a Trello board to manage what needs to be done etc). Then we can decide on what to do next and start making versions of the parsers and the specification, testing the spec with the parsers in our individual projects as we go along.

@rob42 my vote's for Signal K for the project/organisation and "KJSON" as the name for the format (mainly to distinguish between JSON and KJSON when we're talking about stuff).

[fabdrol]

@tkurki btw, a few notes on client side:

• web components are indeed still in a very early stage of implementation and mainly work in newer browsers (see http://www.polymer-project.org/resources/compatibility.html). However, since most Consumer/client implementations would be for a computer, tablet etc on a boat (and not the general public on the internet), one could be quite picky about which browsers they support (and besides, most tablets and phones are webkit-based anyway so that's not really an issue).
• AMD: I hate it's syntax and I personally would never use it. But, that's a choice, of course.
• Browserify: doesn't do on-demand loading of scripts but minifies them all into a big bundle file. Not a problem in our case as bandwidth is plenty. Main upside: very nice syntax (just do require('module-name') and browserify does the rest) and the ability to use Node.js modules on the browser. Actually, most modules on NPM "Just Work" in the browser!

But, this is all a bit off-topic - as @ktuukkan pointed out. The Consumer implementation is entirely up to the developer's preferences, I suppose.

[timmathews]

Consider this my vote for SignalK as well!

[keesverruijt]

+1 for SignalK.

2014-04-01 14:06 GMT+02:00 Tim Mathews notifications@github.com:

Consider this my vote for SignalK as well!

— Reply to this email directly or view it on GitHubhttps://github.com/tkurki/navgauge/issues/4#issuecomment-39197599 .

[timmathews]

In the interest of getting sh*t done, github.com/SignalK is now a project. You should have all gotten notification in one form or another.

EDIT: I meant organization of course

Now, @rob42 since you came up with this name and mentioned a logo, we need a logo. :smile:

[timmathews]

One more thing before I'm off to work. When do we ask Ben at Panbo to blog about us? The resultant conversation should be, erm, interesting.

[tkurki]

Get something that actually interoperates done first? Or something that has been impossible/hard/expensive without SignalK.

Speaking of communication: this issue is getting a bit longwinded. Ideas on how to move forward?

tiistai 1. huhtikuuta 2014 Tim Mathews notifications@github.com kirjoitti:

One more thing before I'm off to work. When do we ask Ben at Panbo to blog about us? The resultant conversation should be, erm, interesting.

Reply to this email directly or view it on GitHubhttps://github.com/tkurki/navgauge/issues/4#issuecomment-39200557 .

[fabdrol]

How about IRC + Trello for managing tasks/features/whatever?

[timmathews]

Perhaps a google group as well?

[keesverruijt]

I don't mind IRC but I don't have a reader open for that normally, so I'd prefer a service that allows reply-via-email or supports XMPP.

Do we want privacy?

2014-04-01 16:11 GMT+02:00 Tim Mathews notifications@github.com:

Perhaps a google group as well?

— Reply to this email directly or view it on GitHubhttps://github.com/tkurki/navgauge/issues/4#issuecomment-39209514 .

[keesverruijt]

BTW,

+1 for Google Group +1 for Trello, it uses Kanban which I am a great fan of.

[fabdrol]

Okay, let's turn this around. Which services are you guys on already - apart from Github and e-mail. I'm on:

• Jabber (google talk), but poor support for groups
• e-mail (but that get's cluttered to quickly)
• IRC
• Skype

[keesverruijt]

Same here except for IRC.

BTW I am looking for a new cross service chat client anyway, preferably with iOS and Android support.

On 01 Apr 2014, at 16:39, Fabian Tollenaar notifications@github.com wrote:

Okay, let's turn this around. Which services are you guys on already - apart from Github and e-mail. I'm on:

Jabber (google talk), but poor support for groups e-mail (but that get's cluttered to quickly) IRC Skype — Reply to this email directly or view it on GitHub.

[fabdrol]

I did see something interesting a while a go, a chat-based project manager with integration for Github and other services. Can't remember the name though, and if it was free. I'll check it out.