Closed gmegan closed 5 years ago
Thank you for the issue.
I want to try to reproduce in my environment. How did you install udmabuf.ko? If you installed using devicetree, please tell me the contents. Also, if you installed using insmod command, please tell me the arguments.
Thank you for responding so quickly. I build udmabuf.ko in the repo directory and then install with insmod. Here are my commands to build the module and tests:
$ git clone https://github.com/ikwzm/udmabuf
$ cd udmabuf
$ make
$ cat /etc/udev/rules.d/99-udmabuf.rules
KERNEL=="udmabuf[0-9]*", GROUP="wheel", MODE="0660"
$ sudo insmod udmabuf.ko udmabuf0=1048576
$ gcc udmabuf_test.c -o udmabuf_test.x
$ ./udmabuf_test.x
uum...
I have an Ultra96. It is powered by Xilinx's ZynqMP. The CPU of ZynqMP is ARM64. I put Linux-Kernel 4.14 on this board. Please refer to this URL for details.
Here is the result of running udmabuf on this system.
root@debian-fpga:~/tmp# git clone https://github.com/ikwzm/udmabuf
Cloning into 'udmabuf'...
remote: Enumerating objects: 25, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 396 (delta 11), reused 19 (delta 6), pack-reused 371
Receiving objects: 100% (396/396), 239.23 KiB | 118.00 KiB/s, done.
Resolving deltas: 100% (220/220), done.
root@debian-fpga:~/tmp# cd udmabuf
root@debian-fpga:~/tmp/udmabuf# make
make -C /lib/modules/4.14.0-xlnx-v2018.2-zynqmp-fpga/build ARCH=arm64 CROSS_COMPILE= M=/root/tmp/udmabuf modules
make[1]: Entering directory '/usr/src/linux-headers-4.14.0-xlnx-v2018.2-zynqmp-fpga'
CC [M] /root/tmp/udmabuf/udmabuf.o
Building modules, stage 2.
MODPOST 1 modules
CC /root/tmp/udmabuf/udmabuf.mod.o
LD [M] /root/tmp/udmabuf/udmabuf.ko
make[1]: Leaving directory '/usr/src/linux-headers-4.14.0-xlnx-v2018.2-zynqmp-fpga'
root@debian-fpga:~/tmp/udmabuf# insmod udmabuf.ko udmabuf0=1048576
[ 132.536923] udmabuf: loading out-of-tree module taints kernel.
[ 132.547518] udmabuf udmabuf0: driver version = 1.4.1
[ 132.552440] udmabuf udmabuf0: major number = 243
[ 132.557202] udmabuf udmabuf0: minor number = 0
[ 132.561788] udmabuf udmabuf0: phys address = 0x0000000070100000
[ 132.567867] udmabuf udmabuf0: buffer size = 1048576
[ 132.572983] udmabuf udmabuf0: dma coherent = 0
[ 132.577586] udmabuf udmabuf.0: driver installed.
root@debian-fpga:~/tmp/udmabuf# gcc udmabuf_test.c -o udmabuf_test.x
udmabuf_test.c: In function 'check_buf_test':
udmabuf_test.c:63:7: warning: implicit declaration of function 'write' [-Wimplicit-function-declaration]
write(fd, attr, strlen(attr));
^~~~~
udmabuf_test.c:64:7: warning: implicit declaration of function 'close' [-Wimplicit-function-declaration]
close(fd);
^~~~~
udmabuf_test.c:71:7: warning: implicit declaration of function 'gettimeofday' [-Wimplicit-function-declaration]
gettimeofday(&start_time, NULL);
^~~~~~~~~~~~
udmabuf_test.c: In function 'main':
udmabuf_test.c:118:7: warning: implicit declaration of function 'read' [-Wimplicit-function-declaration]
read(fd, attr, 1024);
^~~~
udmabuf_test.c:145:23: warning: implicit declaration of function 'lseek' [-Wimplicit-function-declaration]
long last_pos = lseek(fd, 0, 2);
^~~~~
udmabuf_test.c:148:9: warning: implicit declaration of function 'exit' [-Wimplicit-function-declaration]
exit(-1);
^~~~
udmabuf_test.c:148:9: warning: incompatible implicit declaration of built-in function 'exit'
udmabuf_test.c:148:9: note: include '<stdlib.h>' or provide a declaration of 'exit'
root@debian-fpga:~/tmp/udmabuf# ./udmabuf_test.x
phys_addr=0x70100000
size=1048576
check_buf()
sync_mode=0, O_SYNC=0, time = 0.340763 sec
sync_mode=0, O_SYNC=1, time = 0.340823 sec
sync_mode=1, O_SYNC=0, time = 0.340724 sec
sync_mode=1, O_SYNC=1, time = 2.284912 sec
sync_mode=2, O_SYNC=0, time = 0.340711 sec
sync_mode=2, O_SYNC=1, time = 2.284837 sec
sync_mode=3, O_SYNC=0, time = 0.340662 sec
sync_mode=3, O_SYNC=1, time = 2.284818 sec
sync_mode=4, O_SYNC=0, time = 0.340748 sec
sync_mode=4, O_SYNC=1, time = 0.340682 sec
sync_mode=5, O_SYNC=0, time = 2.285120 sec
sync_mode=5, O_SYNC=1, time = 2.284539 sec
sync_mode=6, O_SYNC=0, time = 2.284510 sec
sync_mode=6, O_SYNC=1, time = 2.284424 sec
sync_mode=7, O_SYNC=0, time = 2.284612 sec
sync_mode=7, O_SYNC=1, time = 2.284490 sec
clear_buf()
sync_mode=0, O_SYNC=0, time = 0.025417 sec
sync_mode=0, O_SYNC=1, time = 0.025395 sec
sync_mode=1, O_SYNC=0, time = 0.025388 sec
sync_mode=1, O_SYNC=1, time = 0.026659 sec
sync_mode=2, O_SYNC=0, time = 0.025400 sec
sync_mode=2, O_SYNC=1, time = 0.026655 sec
sync_mode=3, O_SYNC=0, time = 0.025393 sec
sync_mode=3, O_SYNC=1, time = 0.026652 sec
sync_mode=4, O_SYNC=0, time = 0.025401 sec
sync_mode=4, O_SYNC=1, time = 0.025411 sec
sync_mode=5, O_SYNC=0, time = 0.026659 sec
sync_mode=5, O_SYNC=1, time = 0.026659 sec
sync_mode=6, O_SYNC=0, time = 0.026709 sec
sync_mode=6, O_SYNC=1, time = 0.026668 sec
sync_mode=7, O_SYNC=0, time = 0.026654 sec
sync_mode=7, O_SYNC=1, time = 0.026657 sec
root@debian-fpga:~/tmp/udmabuf# ./udmabuf_test.x
phys_addr=0x70100000
size=1048576
check_buf()
sync_mode=0, O_SYNC=0, time = 0.340782 sec
sync_mode=0, O_SYNC=1, time = 0.340959 sec
sync_mode=1, O_SYNC=0, time = 0.340772 sec
sync_mode=1, O_SYNC=1, time = 2.284743 sec
sync_mode=2, O_SYNC=0, time = 0.340702 sec
sync_mode=2, O_SYNC=1, time = 2.284824 sec
sync_mode=3, O_SYNC=0, time = 0.340644 sec
sync_mode=3, O_SYNC=1, time = 2.284745 sec
sync_mode=4, O_SYNC=0, time = 0.340866 sec
sync_mode=4, O_SYNC=1, time = 0.340752 sec
sync_mode=5, O_SYNC=0, time = 2.284731 sec
sync_mode=5, O_SYNC=1, time = 2.284476 sec
sync_mode=6, O_SYNC=0, time = 2.284510 sec
sync_mode=6, O_SYNC=1, time = 2.284402 sec
sync_mode=7, O_SYNC=0, time = 2.284807 sec
sync_mode=7, O_SYNC=1, time = 2.284480 sec
clear_buf()
sync_mode=0, O_SYNC=0, time = 0.026005 sec
sync_mode=0, O_SYNC=1, time = 0.025991 sec
sync_mode=1, O_SYNC=0, time = 0.025989 sec
sync_mode=1, O_SYNC=1, time = 0.026661 sec
sync_mode=2, O_SYNC=0, time = 0.025982 sec
sync_mode=2, O_SYNC=1, time = 0.026657 sec
sync_mode=3, O_SYNC=0, time = 0.025986 sec
sync_mode=3, O_SYNC=1, time = 0.026667 sec
sync_mode=4, O_SYNC=0, time = 0.026007 sec
sync_mode=4, O_SYNC=1, time = 0.025981 sec
sync_mode=5, O_SYNC=0, time = 0.026658 sec
sync_mode=5, O_SYNC=1, time = 0.026659 sec
sync_mode=6, O_SYNC=0, time = 0.026655 sec
sync_mode=6, O_SYNC=1, time = 0.026651 sec
sync_mode=7, O_SYNC=0, time = 0.026664 sec
sync_mode=7, O_SYNC=1, time = 0.026648 sec
root@debian-fpga:~/tmp/udmabuf#
As this result shows, this system could not reproduce your problem.
Please tell me how the cma area is secured. When you boot Linux Kernel, you will see the following message first.
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.14.0-xlnx-v2018.2-zynqmp-fpga (ichiro@sphinx-vm-ubuntu) (gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)) #1 SMP Fri Aug 10 11:05:02 JST 2018
[ 0.000000] Boot CPU: AArch64 Processor [410fd034]
[ 0.000000] Machine model: ZynqMP Ultra96
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] cma: Reserved 256 MiB at 0x0000000070000000
Ah, good question. It appears on this system we do not have CMA area initialized.
$ dmesg | grep cma
[ 0.000000] Memory: 267434048K/268366912K available (7932K kernel code, 1836K rwdata, 3328K rodata, 1280K init, 7719K bss, 932864K reserved, 0K cma-reserved)
udmabuf uses dma_alloc_coherent () to allocate memory. dma_alloc_coherent () allocates memory from kernel space when there is no CMA. When memory is allocated from kernel space, issue #22 points out that there is a problem with fault processing of udmabuf. Maybe this is the cause.
That sounds likely to me at this point. For our case, we actually need memory that is not marked VM_IO or VM_PFNMAP, or we cannot do what we are wanting to do, so the suggested solution in the other issue would cause problems. I will try to enable a CMA area for our system to test.
I configured our system with 512 MB reserved for cma. The test program is now working! Thank you.
I released it temporarily with v1.4.2-rc1. I will confirm the operation as it is for a while.
Thank you for your issue.
I have released udmabuf v1.4.2 which fixes this problem. Thank you.
We are attempting to use the udmabuf device on an aarch64 system. When running the udmabuf_test.c program, it will succeed the first time. The second time we run the program, the kernel will issue errors and the system will go down.
There are many of similar error messages.
dmesg: