USB protocol - Packet Tearing

[andredunford]

LibreVNA Version

LibreVNA USB Protocol v12

Steps to reproduce

• establish connection to VNA via USB interface
• send SweepSettings packet with valid settings
• read VNADatapoint packets in a loop
• evaluate the actual length of packets compared to expected length

Alternatively

• start LibreVNA GUI
• launch USB monitoring application (e.g. Wireshark)
• initiate sweeps and observe packets for irregular response lengths

Expected behavior

Point data should be received as coherent packets (i.e. from Header byte 0x5A through to the CRC bytes 0x00 0x00 0x00 0x00

Extra information & Setup and Calibration files

When reading VNADatapoint packets via the LibreVNA USB protocol, occasional packets exhibit 'tearing'. This occurs when a packet is sent partially, split across separate buffer transfers.

Initially, I thought this may be related to packet aggregation (discussed elsewhere). I've since been able to determine that aggregation is directly related to higher IFBW and host application read rate. Tearing may have some relation to IFBW, but occurs separately from aggregation, and appears to be device/firmware dependent.

In 'torn' packets, data continuity seems to be conserved, meaning that point data can be reassembled from sequential partial packets if needed. This, of course, requires additional overhead to identify and handle.

Although not a critical issue at the moment, it would be good to identify and address the cause of packet tearing. Once my initial interface is serviceable, I may have an opportunity to dig into the firmware implementation a bit further and try to address it.

[jankae]

I think you just have to work with this. I look at the USB datastream just like at a TCP connection: you don't know how many bytes you will get per read and packet aggregation as well as tearing may happen.

One explanation for the tearing: The LibreVNA has an internal USB buffer. Whenever a packet has to be transmitted, it is encoded first and then the resulting bytes are added to the USB buffer and transmitted in the background. The buffer is implemented as a ring buffer but during actual transmission, the DMA can only handle one continuous segment of data. So when the ring buffer loops around to the beginning, I need to split the transfer into two smaller transfer and that split may just be within a packet.

In my GUI implementation, I try to continuously read into a buffer. Whenever new data arrives, I check if a packet is complete, remove it from the buffer and pass it to the next layer of the software. Feel free to use my implementation as a starting point. You may need to adjust some things, especially if you don't use Qt but the overall packet extraction and parsing should work just fine.

[andredunford]

Thanks Jan; the ring buffer explanation totally makes sense, and I can manage with the current behaviour.

So far, I've implemented separation of aggregates and reassembly of partial packets after each read, based on the header values and packet length fields. There are some edge cases I can think of where this might fail though. Your implementation may be more efficient/rigorous. I'll have a look shortly.

I'll go ahead and close this issue. And If I have some spare time in the future I may look at it again, and see if it would be feasible to check the remaining length to the end of the ring buffer, and then advance to the beginning if it would fall in the middle of the packet.