JuliaComputing / xtrx_julia

XTRX LiteX/LitePCIe based design for Julia Computing
BSD 2-Clause "Simplified" License
1 stars 0 forks source link

DMA buffer management with underflows/overflows #100

Open maleadt opened 1 year ago

maleadt commented 1 year ago

When userspace experiences a lag, e.g. because of a GC pause, the hardware may run out of samples to transmit. In order to avoid transmitting stale data, we have a PR proposing to introduce a TX inhibitor flag that gets toggled when we underrun our buffers: https://github.com/JuliaComputing/xtrx_julia/blob/142de4d947a4b1df407af44ecfa6cee9c6db6f70/software/litepcie-kernel-module/main.c#L594-L601

This inhibitor just disables TX in the PHY, so the DMA keeps running (i.e. hw_count keeps increasing).

Later, when userspace catches up, we re-enable TX: https://github.com/JuliaComputing/xtrx_julia/blob/142de4d947a4b1df407af44ecfa6cee9c6db6f70/software/litepcie-kernel-module/main.c#L811-L816

Of note is that userspace needs to actively catch up, so if the hardware got past us by, say, 10 seconds of data, userspace will have to first submit 10 seconds of data -- which will get ignored -- before the inhibitor will get disabled and we start transmitting actual data again. This may be wanted if we want to maintain some sort of synchronization between these counts and the data, or want to ensure that the data is "continuous" (disregarding the zeros we transmitted). On the other hand, it results in userspace taking quite a while to catch up, and that's why I proposed to quickly get out of this situation by bumping the sw_count up to what the hardware is currently at (avoiding having to transmit 10s of ignored data): https://github.com/JuliaComputing/xtrx_julia/blob/4e0983b417dc3a5c0ac957bb99fb14f3043a817d/software/soapysdr-xtrx/Streaming.cpp#L324-L329

According to @sjkelly, that change is problematic, and got reverted. Note that we are already doing this for the RX path (with the same considerations as above): https://github.com/JuliaComputing/xtrx_julia/blob/4e0983b417dc3a5c0ac957bb99fb14f3043a817d/software/soapysdr-xtrx/Streaming.cpp#L254-L259

Also note that the above synchronization bit might be important when we run on the XYNC and expect the buffers to be aligned (in which case we don't want one XTRX to suddenly jump ahead).