Serial handling at high data rates with HW flow control on ST platforms

[RobMeades]

Description

• Type: Possible Bug?
• Priority: Minor

Target STM

I'm dealing with an application on an ST platform in which the serial port is being run at quite a high rate (460800 bits/s). In order to receive characters reliably at the MCU with this kind of rate, HW flow control is being employed. While investigating some character loss I'm seeing in this scenario I noticed something odd: once the serial port buffer is sufficiently full for the RTS line to be raised, it remains in that state; the RTS line is raised after every character, like the serial port's HW buffer is never being emptied.

Looking at the code from UARTSerial() down:

• UARTSerial::rx_irq() calls SerialBase::_base_getc() in a loop while SerialBase::readable() is true.
• ST's serial_readable() (in targets/TARGET_STM/serial_api.c) just checks for the flag UART_FLAG_RXNE being set for the UART.
• However, the interrupt handler (uart_irq() in targets/TARGET_STM/TARGET_STM32F4/serial_device.c) clears the UART_FLAG_RXNE flag.

So if the interrupt has gone off and flow control has stopped any more characters being received, the flag will not be set and UARTSerial::rx_irq() won't go around its loop at all, which would lead to the buffer never being emptied. I think...?

Aside from the fact that this would be inefficient, I wonder if it might be implicated in the character loss I'm seeing.

Steps to reproduce Run serial port via UARTSerial class (e.g. the new Cellular API) on STM part at a data rate high enough that the interrupt can't be serviced within the usual interrupt latency (e.g. 460800 bits/s) and watch how quickly RTS is raised once a few 10's of characters have been received.

cc: @kjbracey-arm , @hasnainvirk , @adustm , @LMESTM , @jeromecoutant

[kjbracey]

DMA would absolutely make a difference. One or two bytes at a time is wasting a lot of CPU time.

LWIP is escaping frames into a buffer, and so should end up doing a single write(500) for each UDP packet or so for your example. If that write(500) could be programmed as a single DMA transfer you'd have so much more CPU time available.

[RobMeades]

In that case I would like to put in a request to ST for DMA/"multibuffer" support in the STM32F4 serial driver ('cos I still think that is how the chip designer intended the UART to be used, it's quite strange not to have a HW buffer in the UART on such a deeply capable chip).

It seems to me that the simplest solution would be to put this in place inside the HAL's normal serial_getc()/putc() API, which provides an instant advantage, effectively compensating for the lack of a HW buffer and also avoiding the complication of adding asynch support to UARTSerial().

@LMESTM: how should I go about making such a request? Is there something I can do to help it along (testing, prototyping, etc.)?

[kjbracey]

I still don't quite understand what the datasheet means by "multibuffer" - I'm not seeing anything that isn't just the normal programmed DMA. If there was FIFO-like support, then maybe it would be the answer.

Otherwise on simplicity, I think it would probably be simplest for ST to make asynch use DMA, as the UART HAL there does have DMA transfer functions, I believe. That would mean asynch support in UARTSerial, but I don't think it's that complex, at least for transmit.

But I find myself now doubtful about whether the current serial_rx_asynch() API is useful at all for this application. What if you don't know upfront a specific number of bytes to receive? It possibly has that same fundamental flaw as Stream::read().

[RobMeades]

Selfishly, I'd be happy with that as all my problems are transmit oriented.

[LMESTM]

I agree that the first step is to move to asynch API indeed, using DMA with 1 byte transfers wouldn't help at all as what we need to avoid is SW handlers to be called at the same rate as data. So let me know when UARTserial over asynch is available, and I'll see what we can do about using DMA, I've looked at it in the past and the difficulty is about having a generic layer to share DMAs amongst drivers. Of course we can support this specific case and see the possible benefit.

[LMESTM]

@RobMeades is there still a problem here or shall we close ?

If still need for improvements, @kjbracey-arm

That would mean asynch support in UARTSerial,

Is this planned ?

[kjbracey]

My understanding is that HAL API changes in the async area are planned, and I've been advised to hold off on any use of the existing API. I did have a prototype patch for async write that Rob tested, but not planning to move that through at the moment.

Also, my own investigations suggest any attempt to really use DMA for serial is very troublesome on most chips we support, which is presumably why most HALs implement async as PIO...

[RobMeades]

I'm happy to close this and await developments as my immediate need for high data rates has gone away. We will still need higher data rates of course, since modems do support much higher rates these days, but it can wait 3 to 6 months to allow the world to move on/catch-up.