unable to read streaming data from USB device

[insane-dreamer]

I'm trying to read streaming data from an FTDI USB dongle that is connected over BLE to a hardware device which once connected continuously sends out a stream of bytes at 500Hz. I can successfully read data from the device on Windows using the ftd2xx DLL + the ftd2xx python wrapper. But I'm now trying to get it working on Linux and ran into issues with the FTDI .so, and ftd2xx is no longer actively maintained, so we're trying to use pyftdi (thanks!). The dongle reports (through a built-in LED) that it is successfully paired and connected to the device, so data should be coming through.

I'm using the following test script, based on the settings we use ftd2xx on Windows:

from pyftdi.ftdi import Ftdi
ftdi = Ftdi()
devices = ftdi.list_devices()
print(devices)
dev = devices[0][0]
ftdi.open(product=dev.pid, vendor=dev.vid, serial=dev.sn)
ftdi.set_baudrate(1000000)
ftdi.set_line_property(bits=8, stopbit=1, parity='N')
ftdi.set_latency_timer(2)
ftdi.set_flowctrl('hw')
data = ftdi.read_data(1024)
print(data)

result:

[(UsbDeviceDescriptor(vid=1027, pid=24577, bus=1, address=5, sn=<omitted>, index=None, description=<omitted>), 1)]
b''

The FTDI configuration are based on the settings which we use with the FTDI DLLs, including the following: setFlowControl(ftd.ftd2xx.FLOW_RTS_CTS, 0x11, 0x13)

In looking at the pyftdi code, a value of hw is equivalent to RTS/CTS (0x1 << 8) which is equivalent to ftd2xx.FLOW_RTS_CTS (0x100). The other two values in SetFlowControl are the XON and XOFF - I couldn't see any way to set these in pyftdi.

Am I missing a step or doing something wrong? Thanks!

[eblot]

Hi,

I have no idea about the meaning of FTD2xx arguments (I only know I do not want this kind of library :-)). What do these parameters mean?

However, if you just want to use an FTDI device on Linux as a USB-serial bridge, do you really need PyFTDI at all? Linux comes with FTDI kernel drivers for using them with UART, so unless you need some specific PyFTDI features I would recommend to stick with the kernel drivers and open the FTDI UART port as a regular TTY device.

[insane-dreamer]

Thanks for the response!

What do these parameters mean?

I found equivalent settings in pyftdi for all of the ftd2xx settings we use on Windows (provided by the device manufacturer), such as baud rate, latency timer, stopbit, parity bit, etc., except for the flow_control. In FTDI2XX SetFlowControl takes a hex value to indicate whether to use RTS/CTS, XON/OFF or DTR/DSR. Our setting is RTS/CTS. The other two arguments of the SetFlowControl are the XON and XOFF bytes but according to the FTD2XX Prog Guide, these only pertain if XON/XOFF is used, so shouldn't matter. (see screenshot from FTD2XX Programming Guide

From what I understand, Ftdi.set_flowctrl("hw") indicates RTS/CTS, so it should be the same as what we're using with ftd2xx. Does that seem right?

do you really need PyFTDI at all?

Good question. We're looking at pyftdi as a high-level library where we can easily configure and read data from the device (as well as write, but mostly read) on Linux without writing our own code to interface directly with the kernel drivers (that's not really our area of expertise, and unfortunately the device manufacturer doesn't officially support Linux so we're on our own). Should we look for another approach?

[eblot]

From what I understand, Ftdi.set_flowctrl("hw") indicates RTS/CTS, so it should be the same as what we're using with ftd2xx. Does that seem right?

Yes. "Software" flow control is not supported anyway.

Should we look for another approach?

I think so. It is quite easy to work with a TTY device from an application. Once you've configured it - which is a bit circumvoluted at first since TTY device date back from the '70 - you just need to read and write to the device as you would do it with a file. The nice stuff is that is once it is done, you can use whatever UART you want to communication with your BT device: a local UART port, or any other supported USB-UART bridge on the market (there are many of them, and cheaper than the FTDI...)

You can find a lot of example on the web.

Even better: if you want to use Python pyserial module can do everything for you - so you do not need to dig into the TTY legacy mess - and all my other remarks apply: you can use whatever UART port you need.

See https://pythonhosted.org/pyserial/shortintro.html

As a final note: HW flow control in FTDI is fully broken: this is not a SW issue, but a HW issue which is documented in the FTDI chip "application note" and I believe that never got fixed. For some HW bad design in FTDI, it can take up to 3 "overflowed" bytes to update the RTS line (or the CTS line, I never remember which one is which) once the internal FTDI FIFO is full. This means that @ high speed, whatever flow control you select, FTDI device are able to trash up to 3 bytes before asserting the HW flow control line...

[insane-dreamer]

Thank you. We'll investigate using pyserial. Would you recommend pyserial over using pyusb?

[eblot]

pyserial use the Linux kernel interface (and it works on Linux/macOS/Windows crap, the only difference is usually restricted to the port name, e.g. "COM1" vs. "/dev/tty.usbserial.xxx" vs. "/dev/ttyUSB0"), so it is likely to be more responsive than the full pyftdi + pyusb + libusb + kernel stack.

Short answer: if you want to use a UART comm port from Python - whether it is a local port or a remote USB-UART bridge, pyserial is the best module to pick.

[insane-dreamer]

Thanks for all the advice. Much appreciated.