HS38x2 Linux, buildroot, how-to-activate UIO driver.

[joshualin-petaio]

Hi export, Can you help how to activate UIO driver in buildroot? Thanks, Joshua.

[EvgeniiDidin]

Hi Joshua,

Further instructions implies that buildroot was somehow configured(with kernel enabled) and built. There are three steps how-to enable UIO driver in buildroot: 1) Enable UIO in kernel. In terminal in buildroot directory run "make linux-menuconfig". This command will open configuration menu for Linux kernel. After opening press "/" button and enter "uio" in a search field:

Press "1" and enable UIO by pressing "y". Also enter UIO driver submenu and enable next options:

Save and exit menu.

2) Specifying UIO regions in .dts file. In order of specifying UIO regions it is necessary to modify .dts file. To find out which .dts file is used in buildroot configuration please run "make menuconfig" in buildroot root directory. Go to "Kernel" menu and the name of .dts file is defined by next field:

In configuration from the picture above the .dts file is "hsdk.dts"

Next open specified by your configuration .dts file in the following directory:

The UIO section should be specify in subsection where IO devices are defined, in case of hsdk.dts UIO section should be located in soc subsection: The full contents of the file from picture above can be found here: https://elixir.bootlin.com/linux/latest/source/arch/arc/boot/dts/hsdk.dts

The UIO section example:

uioEV: uio@0 {
compatible = "generic-uio";      /* NOTE!!! value should be the same in boot command, see "step 3" */
 reg = < 0x0 0xffff0000 0x0 0x4000        /* 4KiB */            /*  for arconnect*/
         0x0 0x20000000 0x0 0x10000000    /* 256 MiB */    /*  for shared data */
         0x0 0x10000000 0x0 0x08000000>;  /* 128 MiB */   /* for code */
/* It is possible to create more UIO regions if necessary */
 reg-names = "arconnect", 
             "ev_shared_mem",
             "ev_code_mem";
interrupt-parent = <&idu_intc>; 
interrupts = < 7 1 >; };  /* interrupt line #7 */

3) Enabling UIO in Kernel boot command. It is also necessary to enable UIO in Kernel boot command. The boot command is located in the .dts file described in step 2. E.g. bootargs line in hsdk.dts:

bootargs = "earlycon=uart8250,mmio32,0xf0005000,115200n8 console=ttyS0,115200n8 debug print-fatal-signals=1";

Please add a uio_pdrv_genirq.of_id=generic-uio substring in the end of the string, the line should be like:

bootargs = "earlycon=uart8250,mmio32,0xf0005000,115200n8 console=ttyS0,115200n8 debug print-fatal-signals=1 uio_pdrv_genirq.of_id=generic-uio";

Note!!! The value in uio_pdrv_genirq.of_id=generic-uio should be the same as compatible field in UIO section in step 2.

Rebuilding kernel in buildroot After doing all modification just run "make" in buildroot root directory, it will rebuild kernel once. If more changes in .dts or kernel code were performed run "make linux-menuconfig" then exit the menu, this will trigger buildroot to rebuild Kernel on next "make run". Commonly buildroot is not rebuilding packages which were already built.

Verifying UIO in runtime Once image is built and booted to verify that UIO regions are available go to /sys/bus/platform/devices/xxx.uio/ directory. If everything is correct, the directory uio0 with maps[0,1,2,...](maps = regions defined in .dts) directories inside should appear.

Please leave a comment in this issue if you face any issues.

Best regards, Evgeniy

[joshualin-petaio]

Hi Evgeniy,

I can find 0.uio in /sys/bus/platform/devices/xxx.uio/ But can't find maps in /dev/ Do you know why ?

BR, Joshua

[EvgeniiDidin]

Hi Joshua,

Missing "/dev/uio0" and "maps" directory means that UIO is not working properly. Could you please search for "uio" messages in dmesg output and share the result of the search? Here is the result of the search from dmesg from the system with working uio:

# dmesg | grep -i uio
Kernel command line: earlycon=uart8250,mmio32,0xf0005000,115200n8 console=ttyS0,115200n8 debug print-fatal-signals=1 uio_pdrv_genirq.of_id=generic-uio
uio_pdrv_genirq ffff0000.uio: use noncoherent DMA ops

Best regards, Evgeniy

[joshualin-petaio]

Hi Evgeniy,

following is my dmesg content :

Best regards, Joshua

[joshualin-petaio]

Hi Evgeniy,

When I change the some setting in buildroot from success case like enable BR2_ARC_ATOMIC_EXT, uclibc -> glibc, enable c++. It will run linux image fail after downloading. Can you help me to check it ? Attach include buildroot .config, linux .config, linux massage. linux_image_setting_and_log.zip

Thanks, Joshua

[EvgeniiDidin]

Hi Joshua,

I took a look in the boot log and found that Kernel was built against uclibc toolchain:

Linux version 5.10.80 (joshua@localhost.localdomain) (arc-linux-gcc (ARCv2 ISA Linux uClibc toolchain - build 965) 11.2.0, GNU ld (ARCv2 ISA Linux uClibc toolchain - build 965) 2.37) #11 SMP PREEMPT Thu Mar 31 04:58:58 EDT 2022

At the same time buildroot was configured to use Glibc toolchain. This discrepancy may cause the error you have observed.

Please try to do a proper clean in Linux kernel directory and rebuild the image. You can clean with next command:

make clean

Best regards, Evgeniy Didin

[joshualin-petaio]

Hi Evgeniy, I change to use glibc. But it still has this issue. dump log is here. minicom.txt

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

In order to put files in to linux image, I try to use option BR2_ROOTFS_OVERLAY in buildroot. I can see the files are in the /buildroot-dir/output/images/rootfs.tar, but I can't see them after download vmlinux. Do you know how to run this correctly ?

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

I found that files can't put in /tmp/ folder. When I put them to /usr/, I can see then after download vmlinux.

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

When I run openamp host application after download vmlinux. It shows LIB not found. "# ./echo_test
./echo_test: error while loading shared libraries: liblibsysfs.so: cannot open shared object file: No such file or directory " And I try to enable option "BR2_PACKAGE_LIBSYSFS" in buildroot, but it still show this error. Do you know how to enable libsysfs correctly ?

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

I found the linux seems not enable data cache. But we need enable data cache for non-dword alignment address access. Because our HW bus only support DW alignment. So we need use cache region for non-DW alignment address access. And with cache coherency mechanism for openamp. Do you know how to enable data cache in linux ?

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

Some information update. Some ARConnect region can't write but some region can. The 0xaf000000 is can't. But 0xaf000104 is can. (arconnect cmd address)

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

May I use MMIO to test 0xaf00_0000 in the rff* test code ? I try to include and use __iomm and ioremap(). But it always show "implicit declaration of function" and "fatal error: sys/io.h: No such file or directory"

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

Our HW has two bus: CBU and LBU. DRAM region use CBU. ARConnect and UART use LBU. But when linux access ARConnect, it use CBU now. So I got the bus error exceptioin. Do you know how to setup ARConnect use LBU in linux ?

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

There has two question.

1. linux RAM base address. Default use 0x8000_0000 caahe region and run normally. Then we try to use 0x9000_0000 cache region and run normally too. But when we try to use 0x1000_0000 cache region, CPU will get double fault exception in arc_mmu_init(). Does there has some default setting in linux to cause conflict ?
2. linux and openamp use cache region but can't byte access ? We observe when linux use this cache region, always use DW access no matter what FW want to use byte access, like metal_io_write8().

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

I found our HW 0x1 and 0x8 are mapping to the same DRAM address. So CPU stuck when test 0x1 region if linux ram base 0x8. Because the code had be modified. Now there has two issue as following:

1. we can put linux at 0x8 and test 0x1 region but only 4KB !!! If over 4KB, if will get "Invalid Write @ 0x20134000 by insn @ 0x105d4" and "Segmentation fault".
2. if put linux at 0x1, it will get double fault exception after enable MMU (in arc_mmu_init, mmu_setup_asid(NULL, 0);). It always get exception and go into EV_MachineCheck() when execute instruction after enable MMU.

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

Following is error message when I run the assemble code step by step.

0x000105dc 46 in rff-test-uio-generic_forjoshua.c (gdb) stepi 0x000105de 46 in rff-test-uio-generic_forjoshua.c (gdb) stepi 47 in rff-test-uio-generic_forjoshua.c (gdb) stepi 0x000105e4 47 in rff-test-uio-generic_forjoshua.c (gdb) stepi

Program received signal SIGBUS, Bus error. do_access (region=0x20132000, size=3, read=true, io_single=false) at rff-test-uio-generic_forjoshua.c:47 47 in rff-test-uio-generic_forjoshua.c (gdb) stepi potentially unexpected fatal signal 7. Path: /root/test_arconnect_0x104 CPU: 0 PID: 102 Comm: testarconnect Not tainted 5.16.0 #6 Bus Error from Data Mem @off 0x105e6 in [/root/test_arconnect_0x104] VMA: 0x00010000 to 0x00012000 ECR: 0x00011001 EFA: 0xaf000104 ERET: 0x000105e6 STAT: 0x80080082 [IE U ] BTA: 0x2005ae08 SP: 0x5fd95d30 FP: 0x5fd95d48 BLK: 0x108e6 LPS: 0x200896d4 LPE: 0x200896dc LPC: 0x00000000 r00: 0x20132000 r01: 0x00001000 r02: 0x20132104 r03: 0x00000000 r04: 0x5fd95cf0 r05: 0x5fd95884 r06: 0x2007d48c r07: 0x00000001 r08: 0x00000040 r09: 0x00000000 r10: 0x00000000 r11: 0x20132000 r12: 0x20132000 r13: 0x00000003 r14: 0x000a24ac r15: 0x00000000 r16: 0x00000000 r17: 0x000a24bc r18: 0x000a24ac r19: 0x00000001 r20: 0x00000000 r21: 0x00000001 r22: 0x00000001 r23: 0xffffffff r24: 0x5f897f8d r25: 0x2001d724

Program terminated with signal SIGBUS, Bus error. The program no longer exists.

And following is assemble code.

0x000105d8 <+68>: ldb r2,[fp,-20] 0x000105dc <+72>: tst_s r2,r2 0x000105de <+74>: beq_s 16 ;0x105ec <do_access+88> 0x000105e0 <+76>: ld r2,[fp,-4] 0x000105e4 <+80>: ldb_s r2,[r2,0] => 0x000105e6 <+82>: stb r2,[fp,-5] 0x000105ea <+86>: b_s 12 ;0x105f4 <do_access+96> 0x000105ec <+88>: ld r2,[fp,-4] 0x000105f0 <+92>: stb 8,[r2] 0x000105f4 <+96>: ld r2,[fp,-4] 0x000105f8 <+100>: add_s r2,r2,1 0x000105fa <+102>: st r2,[fp,-4] 0x000105fe <+106>: ld r2,[fp,-16] 0x00010602 <+110>: sub_s r2,r2,0x1

Thanks, Joshua

[joshualin-petaio]

Hi Evgeniy,

Following is error message when I run the assemble code step by step.

0x000105de 46 in rff-test-uio-generic_forjoshua.c (gdb) stepi 49 in rff-test-uio-generic_forjoshua.c (gdb) stepi 0x000105f0 49 in rff-test-uio-generic_forjoshua.c (gdb) stepi

Invalid Mem Access Path: /root/test_arconnect_0x104 ------------[ cut here ]------------ WARNING: CPU: 1 PID: 89 at kernel/irq/irqdesc.c:679 generic_handle_domain_irq+0xb4/0xc4 Modules linked in: CPU: 1 PID: 89 Comm: testarconnect Not tainted 5.16.0 #3

Stack Trace: arc_unwind_core+0xf4/0x128 dump_stack_lvl+0x2c/0x40 __warn+0x8a/0xc8 warn_slowpath_fmt+0x44/0x8c generic_handle_domain_irq+0xb4/0xc4 arch_do_IRQ+0x3e/0x60 ret_from_exception+0x0/0x8 ---[ end trace f8c996eeae18b6df ]--- CPU: 1 PID: 89 Comm: testarconnect Tainted: G W 5.16.0 #3 Bus Error from Data Mem ECR: 0x00011001 EFA: 0xaf000104 ERET: 0x8056a764 STAT: 0x80081802 [IE K ] BTA: 0x2005ae08 SP: 0x815edf88 FP: 0x5f9d7d48 BLK: 0x108e6 LPS: 0x200896d4 LPE: 0x200896dc LPC: 0x00000000 r00: 0x20132000 r01: 0x00001000 r02: 0x20132104 r03: 0x00000000 r04: 0x5f9d7cf0 r05: 0x5f9d7884 r06: 0x2007d48c r07: 0x00000001 r08: 0x00000040 r09: 0x80081802 r10: 0x00090001 r11: 0x2005ae08 r12: 0x000105f4 r13: 0x80081802 r14: 0x8056a764 r15: 0x00000000 r16: 0x00000000 r17: 0x000a24bc r18: 0x00000000 r19: 0x200896d4 r20: 0x200896dc r21: 0x000108e6 r22: 0x2005ae08 r23: 0x00090001 r24: 0x80081802 r25: 0x00000040

Stack Trace: EV_Trap+0x98/0xd0

And following is assemble code.

0x000105d8 <+68>: ldb r2,[fp,-20] 0x000105dc <+72>: tst_s r2,r2 0x000105de <+74>: beq_s 16 ;0x105ec <do_access+88> 0x000105e0 <+76>: ld r2,[fp,-4] 0x000105e4 <+80>: ldb_s r2,[r2,0] 0x000105e6 <+82>: stb r2,[fp,-5] 0x000105ea <+86>: b_s 12 ;0x105f4 <do_access+96> 0x000105ec <+88>: ld r2,[fp,-4] 0x000105f0 <+92>: stb 8,[r2] 0x000105f4 <+96>: ld r2,[fp,-4] 0x000105f8 <+100>: add_s r2,r2,1 0x000105fa <+102>: st r2,[fp,-4] 0x000105fe <+106>: ld r2,[fp,-16] 0x00010602 <+110>: sub_s r2,r2,0x1 0x00010604 <+112>: st r2,[fp,-16] 0x00010608 <+116>: ld r2,[fp,-16] 0x0001060c <+120>: tst_s r2,r2 0x0001060e <+122>: bne_s -52 ;0x105d8 <do_access+68> 0x00010610 <+124>: ldb r2,[fp,-24] 0x00010614 <+128>: tst_s r2,r2 0x00010616 <+130>: beq_s 50 ;0x10646 <do_access+178> 0x00010618 <+132>: ldb r2,[fp,-20] 0x0001061c <+136>: tst_s r2,r2

MDB screen shot as attach.

Thanks, Joshua

[EvgeniiDidin]

Hi @joshualin-petaio ,

Could you please answer next two questions:

1) According the bootlog it seems that nsimosci_hs_idu.dts file is used in your configuration. Is there any particular reason for using exactly this configuration? 2) I received a screenshot which demonstrates the memory contents after applying my patch(add write 0xdeadbeef to 0xaf000100 in ctx_sw). Do you have an explanation why first byte "0xde" is not presented in the memory?

Thanks, Evgeniy

[joshualin-petaio]

Hi Evgeniy,

1. I'm not found the information about the .dts file. But I remember that dts file is fpga_hs_idu.dts. Can you check it or point out where I can found nsimosci_hs_idu.dts in the boot log ?
2. I use bare-metal code also have this phenomenon. I'll check it with HW team.

Thanks, Joshua