Improve communication between the PC and the car

[DocGarbanzo]

Improve sending data and commands from/to the car

Current state

As of now we have the following connections between the car and the PC

• shell-based rsync to copy data back and forth, shell based ssh in order to drive the car on Linux / OSX. Windows has some similar tools
• UI car screen: This is based on shell subprocesses calling the above Linux / OSX commands. It does not work on windows.
• Webserver access. This is a stand-alone component by @sctse999 which allows to communicate with the car.

Proposals / Options

Optimally we want a solution that works on all platforms and is fast / efficient:

• rsync is probably the fastest way to share data across a network but doesn't have native windows support
• We could integrate the web server into DC and replace the rsync/ssh shell based communication. Alternatively we could explore grpc (https://grpc.io). The web server is written in django whereas we are using tornado within DC. Probably worth to port the code to prevent having to support two different web server technologies. Tornado is a more modern design.

[Ezward]

We are currently running a webserver on the car with an api that allows us to command the car through a rest API and/or websocket. We have an api that allows the car to be calibrated remotely. We have an api that streams from the car's camera. It's running on a Tornado async webserver.

It does not sync data as it is collected. That is something we could add, possibly as a websocket so that the car can push the data to the server.

Is there something else that we want beyond what the car already provides and the ability to push data to the server on the training laptop?

[DocGarbanzo]

We are currently running a webserver on the car with an api that allows us to command the car through a rest API and/or websocket. We have an api that allows the car to be calibrated remotely. We have an api that streams from the car's camera. It's running on a Tornado async webserver.

It does not sync data as it is collected. That is something we could add, possibly as a websocket so that the car can push the data to the server.

Is there something else that we want beyond what the car already provides and the ability to push data to the server on the training laptop?

What @sctse999 added was also the ability to drive the car and to transport tub data from and pilot data to the car. This is pretty much what I added to the UI as well. It basically allows you to run the car without opening any remote shell on it. I have to say that my implementation in the UI is pretty crude, it is based on shell commands like ssh and rsync and text parsing of those. The latter one is very efficient for pushing data through the local network, though. @tikurahul also suggested to look at https://grpc.io as an alternative.

[sctse999]

As discussed in the maintainer's meeting, I will add instruction to the doc how to install the donkeycar console. I will do it tonight.

[Ezward]

I can get behind the goal of running the car without needing to ssh into it or transferring data via scp or a thumbdrive. GRPC is a wrapper around protobuf; it is a very efficient schema's binary protocol with code generation from schemas. I believe we can run this over a websocket. This could clean up our webui code as well; we could refactor all the various api to work using the GRPC connection. This could open up more community contributions as well if we can rationalize the communication between car and client.

[tikurahul]

GRPC over web sockets is a really good idea. I would make sure to work on the scheme first, to get the protocol right. Once you have that, it should be relatively straightforward to actually implement the client and server.