Raw vs Packetized Wireless Messages

[arandell93]

Today I learned.

An interesting idea, perhaps we may require this when we begin pushing large amounts of traffic over the link? @whymarrh thoughts?

[whymarrh]

The SX1231 can be used in a 'raw rx/tx' mode where it just modulates incoming bits from pin #2 and sends them on the radio, however there's no error correction or addressing

I'm going to need you to ELI5. What is the SX1231? The bit about modulating incoming bits—does that mean that there exists the option to stream data off the wireless module onto the serial line? The lack of error correction could be remedied at a higher level, that's fine. If there is no addressing, does that mean it broadcasts the data to all nodes in range? Does the wireless module still offer "encryption" in the raw mode?

[arandell93]

SX1231 is the name of the RFM69HCW's radio chip itself.

From the manual:

[arandell93]

[arandell93]

There's a forum post on the LowPowerLab website, the folks who made the library we're using.

[whymarrh]

While on the surface this sounds like a nice idea, the following seems to be a subtle disclaimer:

Continuous mode: each bit transmitted or received is accessed in real time at the DIO2/DATA pin. This mode may be used if adequate external signal processing is available.

I'll have a read through the datasheet(s) and see what I can and can't understand about this mode.

[arandell93]

True. From what I recall of reading through it, it noted that things like the detection of corrupt messages much harder, since you don't have a set number of bytes nor do you have the acknowledgement bit. However, we could control it within our own API with CRC and leading bytes containing the length of packet. Thoughts?

[whymarrh]

Alrighty, I've read through the SX1231 Transceiver Datasheet and I'm going to cc @FifoIronton on this.

I think this is a discussion better suited for you two, 'cause I don't yet understand 90% of what I read.

[whymarrh]

To answer your question though, I don't yet see how adding our own CRC and length-prefix to messages and using the continuous mode is better than the existing packet mode. Not to mention that the library we're using for the wireless link doesn't support it—are we considering interfacing with the RFM69HCW ourselves? What does using this mode offer us over packet mode?

(Also, there's an "unlimited length packet format described in section 5.5.2.3 that might be an alternative:

5.5.2.1. Fixed Length Packet Format The length of the payload is limited to 255 bytes if AES is not enabled else the message is limited to 64 bytes (i.e. max 65 bytes payload if Address byte is enabled).

And "255 bytes ought to be enough for anybody.")

If we're looking to increase our transfer size, I think we should disable AES and set our PayloadLength to 255. If nothing else, we don't need to encrypt our data.

[arandell93]

Not to mention that the library we're using for the wireless link doesn't support it—are we considering interfacing with the RFM69HCW ourselves?

To do this using the raw format would be much simpler than using packetized, it bypasses SPI and a bunch of back-and-forth. We could make it work.

If we're looking to increase our transfer size, I think we should disable AES and set our PayloadLength to 255. If nothing else, we don't need to encrypt our data.

This was another thing I came across that looks like it could be an improvement for us. Quadrupling our packet size would be an awesome way to get around some inevitable issues. 255 bytes would certainly be enough for us until we upgrade to fancier hardware. However, if we do this, we will also quadruple the time it takes to send a message (to some 400 ms at the current bitrate) over the wireless link. We need to do some experimentation with range (#21) before we can make any changes.

[FifoIronton]

Although I only had a brief look at things, the end goal of using the constant transmit mode is to create a bridge between the two microcontrollers that doesn't have an SPI interface, right?

Will writing the code for this actually be easier than writing the code for the current interface? Right now, the packets get stored in a buffer and we have some freedom for when we read them and do things about them. If we bypass all of that and go to continuous mode, we'll have to use our own system for essentially doing the same thing - except on the microcontroller end. When a message comes in, we'll have to interrupt everything on the microcontroller, take the message in, and then process it. We still would need to wait for a checksum at the end of the packet, so we couldn't act on any of the data until the message was done.

Besides a few headaches around the SPI interface, where else do we improve by switching to raw data? If we want to cut down on wasted space in packets and make the latency lower we could look into the "Unlimited Length Packet" mode Whymarrh mentioned, which lets you transmit arbitrary packet sizes (including smaller than 255 bytes), and chuck our own CRC on the end of those messages

[arandell93]

the end goal of using the constant transmit mode is to create a bridge between the two microcontrollers that doesn't have an SPI interface, right?

While it would remove the SPI requirement, it is not the primary driver for exploring this idea. We are mostly doing it to look for ways we can preemptively optimize bandwidth and decrease latency (LakeMaps/boat#47).

packets get stored in a buffer and we have some freedom for when we read them and do things about them.

Good point. We'd need to be diligent with managing data if we were to do this ourselves.

transmit arbitrary packet sizes (including smaller than 255 bytes), and chuck our own CRC on the end of those messages

I think everything in this discussion is leading to using this alternate 'unlimited' packet mode. However, to me it only seems to be beneficial (particularly from a latency point of view) if we can indeed define arbitrarily long packets and not constrain ourselves to 61 or 255 or any fixed length. Otherwise for the majority of messages we're wasting a significant amount of time transmitting zeroes.

Perhaps this issue can be closed and we can open a new issue for defining a new method for the API by which we can define an arbitrary packet length, regardless of AES or Unlimited mode.

[whymarrh]

Perhaps this issue can be closed and we can open a new issue for defining a new method for the API by which we can define an arbitrary packet length, regardless of AES or Unlimited mode.

I think such an API change would be a breaking change in the protocol and may be best left for 2.0.