Setting Connection Endpoints as UDP and HTTP/UWS

[Weffe]

Hello,

I have been heavily considering Deepstream for a project coming up that will be a real-time application. Some background info as to what the details of the project are is that I am part of a team that competes in the University Rover Challenge and we basically build our own mars rovers from the ground up. This year one of the things we are focusing on improving is diagnostics of the rover itself so that means more data to pump through our network to get all the nitty gritty details of whats happening within our system. We control movement through a joystick and would send those commands via a UDP server and off to the rover so that it could move around. Reasons to choose UDP was that we didn't care if we lost mobility packets and just needed it to be responsive and fast. However, when collecting sensor data we would send those back over HTTP to ensure we captured every bit of sensor data. I then used Socket.io to stream sensor data that we would receive off to a React web app to display it for us. So overall, we would be connecting to two different servers (UDP for mobility and HTTP for sensitive data) in order to transfer data back and forth when we run the rover.

Now my idea is that I could possibly use Deepstream as the only entry point for data transfer rather than 2 entry points. What I was thinking was I could spin up my own UDP server (most likely in NodeJS) and pass the port/info as a Connection Endpoint in the settings along with the default websocket/http info. Then, when I want to transfer mobility packets Deepstream can use UDP but when I need to transfer sensitive data that would be done via HTTP. Is this possible at all? What would the setup be like if I were to do this? Is my understanding of Deepstream correct?

The ideal benefit of this would be displaying what mobility commands were just sent on the UI (HTTP) while simultaneously sending those mobility commands off to the rover (UDP) to be executed.

For example:

// file: joystick.js
const joystick = require('joystick')
const deepstream = require('deepstream.io-client-js')
const client = deepstream('localhost:6020').login()

joystick.on('axis', (data) => {
  client.event.emit('joystick_axis_change', JSON.stringify(data))
})

// file: rover.js
const mobility = require('some-mobility-executor-pkg')
const deepstream = require('deepstream.io-client-js')
const client = deepstream('localhost:6020').login()

client.event.subscribe('joystick_axis_change', (data) => {
  mobility.execute(data)
})

// file: webapp.js
const deepstream = require('deepstream.io-client-js')
const client = deepstream('localhost:6020').login()

class App extends React.Component {
  componentDidMount() {
    client.event.subscribe('joystick_axis_change', (data) => {
      this.setState({mobilityCommand: data})
    })
  }

  render() {
    return <div>Mobility command: {this.state.mobilityCommand}</div>
  }
}

This way all data will be passed to Deepstream who will handle all the data transfer. Thanks for reading through my lengthy post 😅

[yasserf]

hey, interesting use case! Before I go into any detail, am I right to assume that when you say HTTP you mean websockets? Got that impression from your example code.

[Weffe]

Yes sorry I meant websockets. Don't think I'll be using the HTTP API very much but I could!

Edit: Actually now that I think about it I could definitely use the HTTP API. At least sending data from the arduino board.

[yasserf]

Hey, yeah that makes more sense!

In terms of UDP, you would need to create your own client as well that talks UDP and allow the connection endpoint to generate it.

An example endpoint that uses websocketsand and protocol is based on emoticons look like this:

https://github.com/deepstreamIO/deepstream.io-connection-emoticon

Does that make sense? Also since you are using UDP I would be assuming your only going to be using events to send updates to the robot, is that also correct?

[Weffe]

Hmm, I know you mentioned on slack deepstream doesn't support running both UDP and websockets together on 1 instance but I could always have 2 separate instances: 1 for UDP and 1 for Websockets. I can then choose to connect to either or depending on the data I'm sending e.g. mobility commands vs sensor data. Should that work?

[yasserf]

If neither of the servers need to communicate with each other (which in your case seems to be the case) then yup!

It is just worth keeping in mind the clients are based ontop of websockets. If your just doing events you can write a pretty lightweight udp version endpoint, but it won't work out the box. Are you doing one to one communication or are you controlling multiple machines at the same time?

[Weffe]

Hmm, okay. We're communicating to multiple machines at the same time (e.g. our robotic arm and our rover's motor wheels moving at the same time). I think what we may end up doing is swapping out socket.io and our express server for Deepstream. But still keep our UDP server for mobility and robotic arm packets.

[yasserf]

To be honest, I would probably go with that approach. Its totally feasible to add UDP functionality to deepstream, but it sort of conflicts with alot of the behavior such as subscribing and stuff.

One thing you could do for the lolz, is great a bridge from your udp server to deepstream, which would essentially allow you to then show most the udp values you have going to the robotics show up on the display via websockets. That should be a tiny provider and can make your frontend show quite alot of the relevant data

[Weffe]

You read my mind, I was actually thinking of doing that haha. I was thinking of when a joystick/gamepad button is pressed I can just send it off to both the UDP and Deepstream server and call it a day. I'll see what works for us but thanks for all your help and insight. I can keep you updated on our project if you would like.

[yasserf]

yeah that would be great!

If you are daring enough to use udp multicast you might be able to literally create a connection endpoint that hooks into the UDP broadcast and translates it directly within the server, which means your joystick doesn't even need to connect to deepstream. But I'm assuming a joystick is quite lighweight in code so having a ds client won't hurt either!

[Weffe]

Yes, our home base computer that will be running our servers and processes can handle all of it so it shouldn't be an issue to create another deepstream client. We demand fast transmission but are still in design phase for our next iteration so everything is up in the air right now. I'll bring up your suggestions at our next team meeting. Thanks again!