NVIDIA / jetson-rdma-picoevb

Minimal HW-based demo of GPUDirect RDMA on NVIDIA Jetson AGX Xavier running L4T
Other
157 stars 44 forks source link

Not able to do pinning of cuda on 18.04 LTS on X86 PC #14

Open apexintern1 opened 1 year ago

apexintern1 commented 1 year ago

When I ran the command ./rdma-cuda, I am getting the following error ioctl(PIN_CUDA src) failed: ret=-1 errno=22

Can anyone let me know on how to resolve this issue? Thanks in advance.

pyt-hnu commented 1 year ago

I am facing the same problem as you, have you solved it?

apexintern1 commented 1 year ago

No, I have not solved it. Do you have any ideas?

pyt-hnu commented 1 year ago

No, I have not solved it. Do you have any ideas?

I solved the problem by accident. I also test this program now(./rdma-cuda). But i stuck on this one :(ret = ioctl(fd, PICOEVB_IOC_H2C2H_DMA, &dma_params)). Corresponding to the kernel program in pevb_dma.

apexintern1 commented 1 year ago

How did you solve it, did you change anything ./rdma-cuda cuda code? Because when I removed the if-else condition while allocating memory, make it allocate for DGPU using cudaMalloc(), I didn't this error "ioctl(PIN_CUDA src) failed: ret=-1 errno=22", but I got error while doing dma transfer, just like you got "ret = ioctl(fd, PICOEVB_IOC_H2C2H_DMA, &dma_params)"

pyt-hnu commented 1 year ago

I haven't change anything. But i add some "printf" functions to debug. and i haven't remove the if-else condititon while allocating memory. last, i meet same problem "ret = ioctl(fd, PICOEVB_IOC_H2C2H_DMA, &dma_params)", I figured out the problem.( static int pevb_dma(struct pevb *pevb, bool c2h)),but i'm still checking it out. I think the problem is probably the fpga, because i using a different FPGA.

apexintern1 commented 1 year ago

No I think, if we program the FPGA according to the given instructions, then it should work fine. I am using the FPGA they have specified, still I am getting errors.

pyt-hnu commented 1 year ago

Thanks for your sharing, I think i still need to learn something about fpga and dma. As for the correctness of the program(rdma-cuda), I am a little skeptical about it.

apexintern1 commented 1 year ago

Can you tell me what are all the changes you have done to solve the errors, and tell me exactly where did you change it.

pyt-hnu commented 1 year ago

Can you tell me what are all the changes you have done to solve the errors, and tell me exactly where did you change it.

Is your kernel loaded onto the FPGA? I'm not doing anything special,

apexintern1 commented 1 year ago

Yes, kernel is loaded in my system. It also in-use for the FPGA.

pyt-hnu commented 1 year ago

Have you finished now?

pateldipen1984-nv commented 12 months ago

share the steps you used to compile the applications and picoevb kernel module. does it have cuda installed? How pico fpga and dGPU connected in your x86 system?

xlyzzz commented 6 months ago

I haven't change anything. But i add some "printf" functions to debug. and i haven't remove the if-else condititon while allocating memory. last, i meet same problem "ret = ioctl(fd, PICOEVB_IOC_H2C2H_DMA, &dma_params)", I figured out the problem.( static int pevb_dma(struct pevb *pevb, bool c2h)),but i'm still checking it out. I think the problem is probably the fpga, because i using a different FPGA.

hi, have you changed the fpga board to solve this problem? or use ( static int pevb_dma(struct pevb *pevb, bool c2h), how to use pevb_dma? thanke you