DMA re-design for HAL_ChibiOS

[tridge]

The strategy we use for handling DMA channels in HAL_ChibiOS needs an overhaul. The current strategy is:

• we allocate DMA streams to peripherals at build time using the script libraries/AP_HAL_ChibiOS/hwdef/scripts/dma_resolver.py
• once allocated, a particular peripheral can only use that stream at runtime
• most peripheral types are allowed to share DMA streams with other peripherals. Some types (such as the ADC and UART RX channels) can't share DMA streams as they never release a DMA channel, the resolver is aware of this and allocates an exclusive stream
• hwdef.dat controls allow the developer to specify that a particular peripheral should get exclusive DMA access to a stream
• we use binary semaphores to manage the sharing of streams at runtime, implemented in libraries/AP_HAL_ChibiOS/shared_dma.cpp
• we detect runtime contention for a DMA stream, and the uart driver reduces IO size for transmit to reduce contention time if a stream has contention

The problems with this strategy are:

• at build time the resolver doesn't have enough information to know which types of DMA sharing will cause performance issues and which won't, sometimes it selects sharing arrangements that badly impact performance, such as sharing between a I2C device and a critical SPI device such as an IMU
• the use of binary semaphores means we don't get priority inheritance, which means if an SPI IMU is waiting on an I2C compass the I2C thread does not get raised priority, which means the I2C thread may not be scheduled as soon as we would like, which increases the impact of the DMA sharing. Many compass drivers will do transfers that take well over 1ms on I2C, which severely limits the max loop rate if there is a SPI IMU sharing the DMA channel

To solve this I would like to make the following changes:

• allow for runtime DMA stream assignment. The job of dma_resolver.py will be to get the list of possible DMA streams a peripheral can use. Then a runtime allocator will assign streams from the available set. Runtime assignment will require some tricks with cpp macros, as ChibiOS uses #defines for stream assignment. It should be possible to map these macros into a lookup of a static data structure indexed by an enum of peripherals
• switch to using mutexes instead of binary semaphores in shared_dma.cpp. This means we could no longer release a DMA stream from interrupt context, which means we would need to use events associated with a thread to perform the mutex release. This will have a significant impact on RCOutput (eg. dshot) and UART drivers
• update UART driver to allow it to decide not to use DMA at runtime based on measured contention and baudrate. For baudrates of 115200 and below we should consider switching off DMA for the UART if the contention level is significant
• add reporting of DMA contention via the @SYS filesystem

[peterbarker]

On Tue, 28 Apr 2020, Andrew Tridgell wrote:

• allow for runtime DMA stream assignment. The job of dma_resolver.py will be to get the list of possible DMA streams a peripheral can use. Then a runtime allocator will assign streams from the available set. Runtime assignment will require some tricks with cpp macros, as ChibiOS uses #defines

Are all DMA streams created equal? i.e. is this really just a set of identical resources?

[tridge]

Are all DMA streams created equal?

no, on F4 and F7 specific peripherals can use specific streams. On H7 peripherals are limited to a particular DMA controller, but not a specific stream within that controller

[davidbuzz]

This sounds like a reasonable approach to me, although i'm not sure of the implications of changing from semaphores to mutexes... we use semaphores a lot.... is there an expectation of there being a performance benefit to mutexes ..?