Timeout on reading a Microsoft serial usb device

[martinpalsson]

There seems to be some issues reading from a Serial-USB device.

I get timeout when reading a serial-usb device . I have a devboard making transmissions on uart (on USB) with a few Hz repetition rate. Shows up as Serial USB-unit (COM5, driver: Microsoft Serial-USB driver 10.0.22621.1194) on my PC. I can read COM5 by putty. How ever, after opening a serial port towards COM5 with appropriate settings, when calling port.read the program simply blocks and doesn't return from port.read.

What I have done so far in fault isolation: Serial2-rs seems to work as intended for USB Serial port devices (FTDI driver). Separate serial output on UART periphery, connected to the PC via genuine FTDI UART/USB cable. Shows up as USB Serial port (COM4) on my PC. I can read COM4 by putty, and by the same code as attempted for COM5.

Environment: Windows 11 rustc 1.69.0

[de-vri-es]

Hey, thanks for the report!

Could you tell me a bit more about the physical devices you are using? What brand/model are the serial USB converters (the one that works and the one that doesn't work).

Are they connected to the same UART device when testing?

It seems unlikely that the library would work with one type of serial converter and not the other. But then, serial port syscalls are horrible, so you never know for sure.

[martinpalsson]

Alright. All start from the beginning. Here is a rough sketch of what I'm working on, at least the intention. I have a devboard i can program using arduino. It feeds data to the PC via UART over USB. Works wonders with pyserial, putty etc. But I'm not getting it to work with serial2-rs. I've tried serial2-rs with a separate uart/usb interface cable (FTDI) and it to work as expected with serial2-rs.

I notice that they do not show up the same in the device manager and relies on different drivers, which is to be expected because the "UART over USB" connects the PC with the USB IF peripheral on the MCU, not with a separate USB/UART IF chip. Sorry about the Swedish.

My gut feeling is that there is something amiss with interacting with the MS driver /COM5. I will see if this is really the latest version.

[DanielJoyce]

Pyserial seems to have special support for UART over USB devices?

https://github.com/pyserial/pyserial/blob/31fa4807d73ed4eb9891a88a15817b439c4eea2d/serial/urlhandler/protocol_cp2110.py#L3

Are you using a Silcon Labs adapter?

[de-vri-es]

This problem may have been fixed in 0.1.9. Could you test with 0.1.9 or 0.2.0 to see if it works now?

In 0.2.0, if you are passing a closure to SerialPort::open() as second argument, you will need to call settings.set_raw().

[jarkkojs]

This problem may have been fixed in 0.1.9. Could you test with 0.1.9 or 0.2.0 to see if it works now?

In 0.2.0, if you are passing a closure to SerialPort::open() as second argument, you will need to call settings.set_raw().

How do you construct that closure in practice?

[de-vri-es]

If you're not already passing a closure then don't worry. But it would look something like this:

SerialPort::open("COM3", |mut settings| {
    settings.set_raw()?;
    settings.set_baud_rate(115200)?;
    // Set more settings
    Ok(settings)
})?;

[de-vri-es]

There were changes to timeout settings in 0.2.5. If it still isn't working, please comment and I will re-open the issue.

[jarkkojs]

If you're not already passing a closure then don't worry. But it would look something like this:

SerialPort::open("COM3", |mut settings| {
    settings.set_raw()?;
    settings.set_baud_rate(115200)?;
    // Set more settings
    Ok(settings)
})?;

Thanks. It would be nice to have this snippet also in the documentation.

Just to understand this project better: what made you to develop this given that there exist seriaport crate? Just interested, not opposing.

I recently (bit over a week ago) started zmodem2, which i forked from zmodem because its development had stalled and it would nice to get something better and easier to integrate than lrzsz that everyone uses even in 2023. External program has the problem that interactive tasks such showing progress, canceling the transfer etc. are difficult to implement.

This work is a spin off another project: tior, which I started also during recent weeks. The goals for this project are pretty ground to earth: I just want a basic serial terminal client with a nice user experience for file transfer with progress, cancel etc. niceties. It is right now on halt because my focus is on making zmodem2 polished enough to be usable for this.

So I'm thinking whether I should start using this for tior once I get to back to it.

I work on a lot with SBC's, FPGA's and stuff like that so for me it is pretty common use-case to log in through serial. It is sad that the clients are for the most so bad. I mean during MS-DOS days I used Telemate and Telex and both were superior what exists today, which is kind of ironic :-)

Further (in more distant future) I'm also planning to implement MSLP (aka SMODEM very popular in BBS's during the late days) and use it to get a TTY session with a possibility to a concurrent file transfer so thus I'm looking for a crate with active development.

[de-vri-es]

Hey!

Good point on adding that to the documentation.

The reason for making serial2 was mainly simplicity. The other crates use a trait and different concrete types based on the platform. This crate just has one set of types for all platforms and no custom traits.

serial2 also supports concurrent reads and writes on Windows, which atleast the serial crate doesn't (if I remember correctly). You need to use overlapped IO on Windows to allow that, otherwise, reading from a serial port blocks any write attempt.

[jarkkojs]

Hey!

Good point on adding that to the documentation.

The reason for making serial2 was mainly simplicity. The other crates use a trait and different concrete types based on the platform. This crate just has one set of types for all platforms and no custom traits.

serial2 also supports concurrent reads and writes on Windows, which atleast the serial crate doesn't (if I remember correctly). You need to use overlapped IO on Windows to allow that, otherwise, reading from a serial port blocks any write attempt.

Windows support is actually a selling point for me, as I would like make my stuff work on all major platforms :-)

[de-vri-es]

I do think that serial supports Windows, but I'm not sure if they support concurrent reads/writes from different threads (I think they dont).

But yeah, for me, the main thing was just a simple API with minimal different types and traits. Only one crate instead of three, and no generic S: SerialPort or dyn SerialPort anywhere :)

[de-vri-es]

Added the example: https://docs.rs/serial2/0.2.11/serial2/struct.SerialPort.html#example-2-open-a-serial-port-with-full-control-over-the-settings

Thanks for the feedback!

[de-vri-es]

serial2 also supports concurrent reads and writes on Windows, which atleast the serial crate doesn't (if I remember correctly). You need to use overlapped IO on Windows to allow that, otherwise, reading from a serial port blocks any write attempt.

Note: the serialport crate also doesn't use overlapped I/O on Windows, so reads from one thread block writes from another.

Unless I missed something, this is the only crate using overlapped I/O to allow concurrent reads and writes.

[fko-kuptec]

Unfortunately, I just came across the same issue. I can successfully communicate with my embedded device over UART using hterm or pyserial on W11. With serial2 or any other Rust serial port library that I tested, I can successfully send commands over UART to the device, but I am unable to read back the response. In the case of serial2, I just get a timeout when calling read. I already set the communication to raw using the method mentioned previously in this thread. I don't know, whether this is related to the relative short message sizes in my case (a few bytes), so that Windows maybe doesn't feel urged to return the already received data?

Edit: I am using a genuine FTDI USB adapter with just TXD and RXD connected.

[fko-kuptec]

After having implemented a serial port myself using the windows crate, I found the problem. For some reason, Windows seems to have some weired timeout behaviour, possibly not returning until the timeout is over? Anyways, the solution is to make the read timeout small and to match on the error kind returned by the read method. If the error is ErrorKind::TimedOut, the read can be retried until the data was received. That would look something like this:

// init
let mut port = serial2::SerialPort::open(
        "COM?",
        |mut settings: serial2::Settings| {
            settings.set_raw();
            settings.set_baud_rate(baudrate)?;
            Ok(settings)
        },
    )
    .unwrap();
port.set_read_timeout(Duration::from_millis(5))
    .unwrap();

// read
let mut buffer = [0; 128];
let len = loop {
    match port.read(&mut buffer) {
        Ok(len) => break len,
        Err(error) => match error.kind() {
            std::io::ErrorKind::TimedOut => continue,
            _ => return Err(error),
        }
    }
};

// ...

[de-vri-es]

@fko-kuptec

Thanks for debugging the issue.

But I thought this should all have been fixed in v0.2.5:

- [fix][minor] Mimic Unix behavior on Windows: read will return with available data as soon as possible.

Is it possible that you are you using an older version? Or are you modifying the COMMTIMEOUTS struct directly?

[fko-kuptec]

Unfortunately, I am already using version 0.2.20. Without my hacky fix, it does not seem to work, independent of having the windows feature enabled or not (because I just saw that I forgot).

[de-vri-es]

Ok.. Good to know. If you have the time, can you check if you can reproduce this issue with two USB serial converters connected to eachother (rx to tx, tx to rx)? Because so far I have not been able to reproduce this :(

Maybe it's a buggy serial driver that doesn't implement the Unix still timeout settings correctly :|

The windows feature is only needed for using windows specific functions. If you stick with the normal API you don't need it.

[fko-kuptec]

I will test it out tomorrow and report back :)

[fko-kuptec]

When using two FTDIs, it seems to work fine for me, too. I don't known, whether this is due to some hardware differences or if my test case is not close enough to the real scenario. 🤷‍♂️

[de-vri-es]

Auch.. Thanks for testing though.

Could it be that you're simply seeing "normal" timeouts because the line is silent? What if you increase the timeout instead of decrease it? Or only perform a read after you wrote a command and expect a reply?

Or if you're really waiting for the device to send something, then maybe swallowing timeout errors is the right thing to do.

/edit: Reading back, it seems like this is not what you were describing..

[fko-kuptec]

I think I read after writing the command. Don't know, why this is behaving as is in this specific use-case. But if someone should stumple across the same problem, maybe this thread can help