search

Xilinx / dma_ip_drivers

Xilinx QDMA IP Drivers
https://xilinx.github.io/dma_ip_drivers/
578 stars 420 forks source link

QDMA 2019.1 : Unable to add AXIMM H2C and AXIMM C2H Queues and additional issues as mentioned below #57

Open kewlcreatures opened 4 years ago

kewlcreatures commented 4 years ago

Board: Alveo- U50DD Environment details: Machine Make and Model : x3650 M4 Operating System: Red Hat Enterprise Linux Server release 7.6 (Maipo) Kernel version: 3.10.0-957.el7.x86_64

After make , make install and make install-mods , we attempted to add AXI-MM C2H and AXI-MM H2C Queues. However the queues did not get added with output as below . dmactl qdma8b000 q add idx 0 mode mm dir h2c Zero Qs

On checking dmactl dev list below was the output. qdma8b000 0000:8b:00.0 max QP: 0, -~- However the output should be: qdma82000 0000:82:00.0 max QP: 512, 0~511

On debugging further below are the observations:

  1. root@ XXX.XX.XXX.XX: /sys/bus/pci/devices/0000:8b:00.0/qdma $ cat qmax 0 This should have been automatically populated to 512. We manually updated the same as 'echo 512 > qmax based on the article. https://forums.xilinx.com/t5/PCIe-and-CPM/QDMA-dmactl-can-not-create-queue/td-p/1078463 After that we again attempted to add a queue. However that did not work as well as the other fs fles need to be populated as well.
  2. On further look we could see qdma error in dmesg as below: "qdma:qdma_csr_read: Hardware Feature not supported"

Then taking a deep dive into qdma driver code 2019.1 version we could see the below code in qdma_access.c line number 1943. _if (dev_cap->st_en || dev_cap->mm_cmpt_en) qdma_read_csr_values(dev_hndl, QDMA_OFFSET_C2H_TIMER_CNT, index, count, glbl_tmrcnt); However when I look at the qdma_ip version 3.0 on Vivado 2019.2 I can see that theres no support for AXI-MM with completion infact that option is not available in the main menu. Attempting to fix the issue I've modified the above check to if (dev_cap->st_en || dev_cap->mm_en) in all the qdma code locally. Seems to fix the issues we were facing and the H2C and C2H are up and running. Having said that the dmactl qdma8b000 q list is not yet showing the C2H queue. However need to know if this is the correct direction of the fix as the qdma-driver doesn't seem to be reflecting what the QDMA IP features. By that what I mean is if AXI-MM with completion is not supported by the QDMA IP 3.0 Vivado 2019.1 in hardware how can the code check for the mm_cmpt_en.

Below is the diff of the code fix ive applied locally. Please ignore the Makefile changes. _$ git diff diff --git a/QDMA/linux-kernel/Makefile b/QDMA/linux-kernel/Makefile index 8ffc4c4..62cf6e3 100644 --- a/QDMA/linux-kernel/Makefile +++ b/QDMA/linux-kernel/Makefile @@ -21,6 +21,10 @@ grep = grep 2>/dev/null

ALL subdirectories

ALLSUBDIRS := drv DRIVER_SRC_DIR := drv +#DEBUG := 1 +#DEBUGFS := 1 +# DEBUG_THREADS := 1 +# ERR_DEBUG := 1

subdirectories to be build

SUBDIRS := $(ALLSUBDIRS) diff --git a/QDMA/linux-kernel/libqdma/qdma_access/qdma_access.c b/QDMA/linux-kernel/libqdma/qdma_access/qdma_access.c index cd7aee0..0fc29af 100644 --- a/QDMA/linux-kernel/libqdma/qdma_access/qdma_access.c +++ b/QDMA/linux-kernel/libqdma/qdma_access/qdma_access.c @@ -1805,7 +1805,7 @@ int qdma_set_default_global_csr(void *dev_hndl) qdma_write_csr_values(dev_hndl, QDMA_OFFSET_GLBL_RNG_SZ, 0, QDMA_NUM_RING_SIZES, rng_sz);

hmaarrfk commented 4 years ago

Hey this looks really interesting.

I'm also a user of QDMA with a few patches of my own to share.

would you like to fork this project and merge our changes together?

hmaarrfk commented 4 years ago

I already created my fork: https://github.com/ramonaoptics/dma_ip_drivers

I can make a few small PRs if you want to see the little changes i made to 2019.1

hmaarrfk commented 4 years ago

i pushed a few of our changes on there. our main branch is ours_2019.1

kewlcreatures commented 4 years ago

Hi Mark, Sincere apologies for this delay in replying. I've been very busy. Yes I'll commit my changes to the fork and we can merge our changes together.

BTW with regards to the driver code I do have some more suggestions and questions which could . If you can share your email address I can shoot them to you. Sincere apologies for the delay.

Kind Regards, Vikas L

kewlcreatures commented 4 years ago

Hi Mark, Sincere apologies for this delay in replying. I've been very busy. Yes I'll commit my changes to the fork and we can merge our changes together. BTW with regards to the driver code I do have some more suggestions and questions which could . If you can share your email address I can shoot them to you. Sincere apologies for the delay. Kind Regards, Vikas L

@hmaarrfk fyip

lesjokolat commented 4 years ago

@hmaarrfk

If you like perhaps a potential fix to 512 see some comments from this.

https://github.com/Xilinx/dma_ip_drivers/pull/49#issuecomment-642940357

@bhathaway changes with some tweaks might fix some of that pain problem. I can test for your GIT once any changes made. Im using XDMA btw.

hmaarrfk commented 2 years ago

Hello,

My name is Mark Harfouche. I am not affiliated with Xilinx in any way. Over the years of using QDMA, I've been wanted better community organization.

I've created a fork of dma_ip_drivers which I intend to maintain and work with the community at large to improve.

The fork can be found https://github.com/hmaarrfk/dma_ip_drivers

For now, I am stating the main goals of the repository in https://github.com/hmaarrfk/dma_ip_drivers/issues/2

If you are interested in working together, feel free to open an issue or PR to my fork.

Best,

Mark

hmaarrfk commented 2 years ago

I should really get back to this particular issue.