More Communication Protocols

[djungelorm]

The underlying server communication protocol should be moddable, to allow communication via other means than just google's protocol buffers.

Then we can support, for example:

• http://en.wikipedia.org/wiki/SOAP to make it easy to interact with kRPC from web apps (e.g. JavaScript)
• CCSDS protocol???

[eXigentCoder]

Yeah, I think with npm it should be fine to leave the source where it is. Especially since it's currently working with Travis to run CI & deployment. Once the documentation is done, would we just need to add a link from http://krpc.github.io/krpc to the node documentation file (would most likely be markdown) or would your rather have it as html on that site?

[djungelorm]

I don't mind if you want to host the documentation yourself - probably makes more sense as you could maintain it more easily. And I'm happy to link to it from all the various places.

[djungelorm]

I have now merged the websockets branch into the v0.4.0 branch, and have revamped the UI so now you can create multiple servers, and choose which protocol they use!

Latest build is here: http://krpc.s3-website-us-east-1.amazonaws.com/deploy/v0.4.0/692.1/

And here's a screenshot. The UI is functionally complete, although it could probably do with some visual improvements. Feedback welcome - you can probably tell I am not a UX designer!

[gitanat]

Fantastic news, thanks for all the work djungelorm! Regarding the UI: is there a reason not to have all the servers active all the time? Does it consume a lot of memory / cpu? I would expect most users to not know about TCP/Websockets. Developers will, of course, but many people would just use kRPC to get some external running.

[eXigentCoder]

I was having some issues with the release I was using and like to keep updated so I tried to use the latest 0.4.0 build krpc-0.3.6-73-g42d47a0 since I see the websockets branch has been merged in. However my client no longer connects.

I went back in the websockets release branch and tried out the krpc-0.3.6-69-g49af2e4 build and it seems to have the same problem. krpc-0.3.6-68-ge3e0eab works fine though, any idea what might have changed between the two releases so I can fix my code? Thanks again! My server connections are included in case I'm doing something silly there:

[djungelorm]

Oooo there is a nasty bug! Nothing to do with your client - the UI changes introduced an issue where reconfiguring a server doesn't update its event handlers, leading to it not starting properly.

Should be fixed in this version: http://krpc.s3-website-us-east-1.amazonaws.com/deploy/v0.4.0/706.1/

[eXigentCoder]

Awesome thanks, will test it out this evening!

[nevercast]

Hi @djungelorm

What is the status of WebSockets? Is it implemented and does it just wrap Protobuf?

Regards, NeverCast.

[eXigentCoder]

I've got most of the node.js library working (At least I think so, need to add a bunch more tests), however one part that definitely isn't working is connecting to the stream server.

Am I right in thinking this endpoint isn't upgraded yet to use WebSockets or have I just broken something? The error I get when attempting to connect is:

WebSocket {
     domain: null,
     _events: 
      { error: [Object],
        close: [Object],
        message: [Object],
        open: [Object] },
     _eventsCount: 4,
     _maxListeners: undefined,
     _socket: null,
     _ultron: null,
     _closeReceived: false,
     bytesReceived: 0,
     readyState: 0,
     supports: { binary: true },
     extensions: {},
     _binaryType: 'arraybuffer',
     _isServer: false,
     url: 'ws://127.0.0.1:50001',
     protocolVersion: 13 } } Error: unexpected server response (400)

This is while using the v0.4.0- 711.1 build

[djungelorm]

@nevercast websockets is fully implemented and working in the upcoming v0.4.0 release (pre-release is available with it working). It does indeed use protobuf to serialize messages within the websockets messages.

@eXigentCoder The stream server should work. I'll have a look.

[nevercast]

@djungelorm Thanks excellent for my HTML UI needs. Cheers!

[magwo]

Really looking forward to the websocket support as I'm trying to create a browser-based (for mobile) throttle controller.

The idea is to have the iPhone sitting to the left of my keyboard with a throttle interface, instead of a HOTAS throttle.

[eXigentCoder]

@djungelorm sorry, I was on a bit of a break but am back now and trying to get everything working. I got the latest release of the 0.4 branch (0.4.0-122) and most of the existing code I had is still working except for streams.

I am trying to connect to the stream server using ws://127.0.0.1:50001 but the server returns a 400 Bad Request still.

Does anything there seem obviously wrong to you?

[djungelorm]

@eXigentCoder To connect to the stream server, you need to pass the RPC clients identifier in the connection string (as a base64 encoded string). For example: ws://127.0.0.1:50001?id=8qY/Dfa+lESsDIC2jGat7Q==

This is used to "link" the two connections together in the server side logic. Unfortunately I neglected to add a way for a websockets client to get hold of this id! I've added an RPC called KRPC.GetClientID to allow this.

Here's the latest build with this addition: http://krpc.s3-website-us-east-1.amazonaws.com/deploy/v0.4.0/822.1/

And an example script that connects to the stream server: https://gist.github.com/djungelorm/90fd690db0af5d01f4a31a3b811165b2

[eXigentCoder]

Thanks a lot, your code to get the client Id works perfectly, and I am now able to successfully establish a connection to the stream server!

I noticed in the 0.3x documentation, previously the AddStream method was expecting a Request object:

after taking a look at the source code it looks like it has been changed to take in a ProcedureCall:

So I sent through a ProcedureCall object to add stream and get back a successful response with the id and the stream socket is receiving data however I am unable to decode it. At the moment I'm trying to decode it to a StreamResult:

The buffer library throws the error Illegal group end indicator for Message .krpc.schema.StreamResult: 1 (not a group)

Am I trying to decode to the wrong type maybe?

[djungelorm]

Yeah you are decoding the wrong type. The stream server sends a sequence of StreamUpdate messages, so that's the type you want to decode.

[eXigentCoder]

Ah yes that makes more sense. I updated the code to decode to StreamUpdate but I get a similar error Illegal group end indicator for Message .krpc.schema.StreamUpdate: 1 (not a group).

Sorry to keep pestering you, will keep trying to get it working this side but if you have an idea of where to look I'd appreciate it.

[djungelorm]

There was a bug... The server was writing the size of the message, followed by the message data. I've fixed it to only write the message, so your code should hopefully now work.

Updated version available here: http://krpc.s3-website-us-east-1.amazonaws.com/deploy/v0.4.0/827.1/

And here's the script I used to test it: https://gist.github.com/djungelorm/fb09f4683d8e988eebf4da63a2e340c7 It connects to both servers, sets up a stream for KRPC.GetStatus then prints out all the stream updates that it receives.

And no worries - happy to help!

[eXigentCoder]

Thanks so much, had a quick window to test out and it's working now, time to refactor it to be pretty and easier to use!

[djungelorm]

Quick update from me: I've now "officially" released v0.4.0 with the websockets server included. I also have a serial I/O protocol in the works for release very soon, along with a client library written in C using nanopb targetting embedded devices so you can use kRPC from an Arduino/whatever. I will have another look into adding gRPC support after that is released.

[djungelorm]

There has been some discussion on the discord regarding gRPC. While great to add it in theory, it isn't practical to do so. Here's a summary:

• gRPC requires HTTP/2 (if we want bidirectional streaming, which we do)
• HTTP/2 implementation is not available in Unity
• gRPC also requires .NET Core 3.0 and up which Unity does not support.
• There is an older gRPC implementation that might work, BUT it uses a native library for its transport implementation. This would cause headaches for cross platform support. Plus this implementation is no longer supported.

More info here: https://github.com/grpc/grpc-dotnet/issues/1309