search

xtrx-sdr / images

Pre-built XTRX packages and firmware images
https://www.crowdsupply.com/fairwaves/xtrx
44 stars 27 forks source link

NVIDIA Jetson nano compatibility #48

Open freqmod opened 5 years ago

freqmod commented 5 years ago

Hi

I am trying to make XTRX work with the NVIDIA jetson nano. I have gotten it to work somewhat (if somebody else have problems with lots of messages from PCI add "pci=noaer" to kernel boot), but i still have some problems with DMA buffers.

Due to limited memory i have to reduce the number of DMA buffers and buffer sizes from 32 to 14 in:

xtrxll_base_pcie.c:
-               XTRX_DMA_RX_V0_MAX = BUF_64K,
+               XTRX_DMA_RX_V0_MAX = BUF_32K,

xtrxll_base_pcie.h:
 /* XTRX DMA configuration */
-#define RXDMA_BUFFERS      32
+#define RXDMA_BUFFERS      14

xtrx.c (in the pci driver):
-#define BUFS      32
+#define BUFS      14

//And define:
 #define VA_DMA_ADDR_FIXUP

another option is to keep the buffer amount and reduce the sizes, but i haven't gotten that to work as well as the current approach.

With this i manage to get the test working, and for low samplerate for several tens of seconds. The problem is when i am not able to retrieve the buffers fast enough overflow handling is wrong. 14:44:57.798698 ERROR: [BPCI] PCI:/dev/xtrx0: Incorrect DMA pointers! (bufno=42 bufno_rd=25 rdidx=25 icnt=3)

The question i have is if it is possible to reduce the amount of used DMA buffers without recompiling the FPGA image? If i have to recompile the FPGA image are the sources available?

freqmod@fmjn1:~/Downloads/images/sources/bld$ ./libxtrx/test_xtrx -t -u 100000000 -s 16000000
Master: 128.000 MHz; RX rate: 16.000 MHz; TX rate: 0.000 MHz
RX tunned: 900000000.000000
RX bandwidth: 2000000.000000
RX LNA gain: 15.000000
RX PGA gain: 0.000000
RX TIA gain: 9.000000
RX CYCLES=1 SAMPLES=100000000 SLICE=16384 (PARTS=6103)
PROCESSED RX SLICE 0 /0: res 0 TS:    8192        801 us DELTA    214 us LATE    504 us 16384 samples
PROCESSED RX SLICE 0 /1: res 0 TS:   16384        472 us DELTA     17 us LATE    482 us 16384 samples
PROCESSED RX SLICE 0 /2: res 0 TS:   24576        587 us DELTA     16 us LATE    574 us 16384 samples
PROCESSED RX SLICE 0 /3: res 0 TS:   32768        398 us DELTA     23 us LATE    485 us 16384 samples
PROCESSED RX SLICE 0 /4: res 0 TS:   40960        493 us DELTA     20 us LATE    487 us 16384 samples
PROCESSED RX SLICE 0 /5: res 0 TS:   49152        468 us DELTA     17 us LATE    462 us 16384 samples
PROCESSED RX SLICE 0 /6: res 0 TS:   57344       6230 us DELTA     18 us LATE   6199 us 16384 samples
PROCESSED RX SLICE 0 /7: res 0 TS:   65536         23 us DELTA     21 us LATE   5732 us 16384 samples
PROCESSED RX SLICE 0 /8: res 0 TS:   73728         23 us DELTA      8 us LATE   5251 us 16384 samples
PROCESSED RX SLICE 0 /9: res 0 TS:   81920         23 us DELTA     29 us LATE   4792 us 16384 samples
PROCESSED RX SLICE 0 /10: res 0 TS:   90112         21 us DELTA     12 us LATE   4315 us 16384 samples
PROCESSED RX SLICE 0 /11: res 0 TS:   98304         22 us DELTA      7 us LATE   3833 us 16384 samples
14:44:57.798698 ERROR:  [BPCI] PCI:/dev/xtrx0: Incorrect DMA pointers! (bufno=42 bufno_rd=25 rdidx=25 icnt=3)
14:44:57.798833 ERROR:  [PCIE] PCI:/dev/xtrx0: Got -32!
PROCESSED RX SLICE 0 /12: res -32 TS:  106496       6136 us DELTA      7 us LATE   9465 us 8192 samples
Failed xtrx_recv_sync: -32
RX STAT Overruns:0
Success!
Processed 1 devs, each: RX 2 x 6.347 = 12.695 MSPS (WIRE: 50.778170)    TX 2 x 0.000 = 0.000 MSPS (WIRE: 0.000000 MB/s)
freqmod commented 5 years ago

From reply to email to fairwaves: It seems it's related to https://github.com/xtrx-sdr/images/issues/37 (solution https://devtalk.nvidia.com/default/topic/770634/jetson-tk1/large-coherent-dma-blocks/) Have you tried to increase the coherent DMA pool? It works on most ARM platforms.

-- Using a command line with "pci=noaer vmalloc=512M cma=64M coherent_pool=32M" passes the DMA test, however both test_xtrx and sdrangel does not display any useful data (it does not seem like the buffer is written to from xtrx). However enabling #define VA_DMA_ADDR_FIXUP in xtrx.c in the kernel driver causes proper data to be delivered.

When i get overflows (both with sdrangel and test_xtrx) i get 16:08:26.704863 WARN: [XTRX] PCI:/dev/xtrx0: Total=65536 Processed=0 UserTotal=32768 UserProcessed=0 BUFTS=617644032+0 OURTS=611467264 16:08:26.717247 INFO: [BPCI] PCI:/dev/xtrx0: BUF_OVF TS:618168320 WTS:620153931 WTS_NXT:620167168 TS_NXT:624361472 SKIP 378 buffers INT_S:0 16:08:26.717335 INFO: [CTRL] PCI:/dev/xtrx0: Placing TC @624361472 on 0 data: 0 stat:3d Then i have to reboot to get xtrx to work again.