JuliaComputing / xtrx_julia

XTRX LiteX/LitePCIe based design for Julia Computing
BSD 2-Clause "Simplified" License
1 stars 0 forks source link

HACK: Avoid retransmissions by wiping buffers. #37

Closed maleadt closed 2 years ago

maleadt commented 2 years ago

As per @staticfloat's request, this avoids retransmitting previous buffers by wiping them in the interrupt handler. This is a hacky solution because it's expensive to wipe these buffers, esp. when we'll start using GPUs again (because this would result in writes to GPU memory over the PCIe bus, which I'm not even sure is OK to do in an interrupt handler). But for now, it might help debugging the other issues we're dealing with.

A better solution would be to disable the DMA loop from the interrupt handler when an underflow occurs, and re-enable it in liblitepcie from dma_process or when fetching a new write buffer. But that's more finicky, and I just wanted something that would work for now.

Test script:

#include <stdio.h>
#include <string.h>
#include "liblitepcie.h"

const int data_width = 16;
const char* litepcie_device = "/dev/litepcie0";

int main() {
    static struct litepcie_dma_ctrl dma = {.use_reader = 1, .use_writer = 1};
    dma.loopback = 1;

    if (litepcie_dma_init(&dma, litepcie_device, 1, 0))
        exit(1);

    char *buf_wr;
    char *buf_rd;

    // fill the DMA reader buffers with known data
    while (dma.reader_sw_count <= DMA_BUFFER_COUNT) {
        litepcie_dma_process(&dma);

        // process the buffers that are ready
        while (1) {
            buf_wr = litepcie_dma_next_write_buffer(&dma);
            if (!buf_wr)
                break;

            memset(buf_wr, 1, DMA_BUFFER_SIZE);
        }
    }

    // empty a single buffer to show where we stopped transmitting
    while (1) {
        litepcie_dma_process(&dma);
        buf_wr = litepcie_dma_next_write_buffer(&dma);
        if (buf_wr) {
            memset(buf_wr, 255, DMA_BUFFER_SIZE);
            break;
        }
    }

    // check the looped back buffers
    for (int i = 0; i < 2*DMA_BUFFER_COUNT;) {
        litepcie_dma_process(&dma);

        // process the buffers that are ready
        while (1) {
            buf_rd = litepcie_dma_next_read_buffer(&dma);
            if (!buf_rd)
                break;
            printf("buffer %d: %d\n", i, buf_rd[0]);
            i++;
        }
    }

    return 0;
}

Before this change:

buffer 0: 1
...
buffer 351: 1
buffer 352: -1
buffer 353: 1
...
buffer 541: 1

After:

buffer 0: 1
...
buffer 93: 1
buffer 94: -1
buffer 95: 0
...
buffer 542: 0
maleadt commented 2 years ago

A better solution would be to disable the DMA loop from the interrupt handler when an underflow occurs, and re-enable it in liblitepcie from dma_process or when fetching a new write buffer. But that's more finicky, and I just wanted something that would work for now.

Turns out that doesn't work, the DMA engine expects to be run contiguously and stopping/re-starting it will introduce unwanted behavior/synchronization issues. A better solution would be to introduce a register to disable the RF output when set, which could be toggled by the driver when detecting an underflow.

maleadt commented 2 years ago

The alternative: a CSR that temporarily disables transmission, https://github.com/enjoy-digital/xtrx_julia/commit/9737744842416868bf8c1c9541c30459efa3c4f3. I'll look into integrating this.