arnopo
commented
2 months ago Hello @olneumann Please provide Zephyr and Linux traces to better understand which message is sent or not.
Closed olneumann closed 1 month ago
arnopo
commented
2 months ago Hello @olneumann Please provide Zephyr and Linux traces to better understand which message is sent or not.
tnmysh
commented
2 months ago @olneumann From what's provided in traces (virtio0.rpmsg-tty.257.-1) looks like devices are created at linux side. So Demo is working as expected.
Your concern about following logs:
[ 115.757590] stm32-ipcc 4c001000.mailbox: stm32_ipcc_send_data: chan:1
[ 115.757850] stm32-ipcc 4c001000.mailbox: stm32_ipcc_tx_irq: chan:1 tx
[ 115.757906] stm32-ipcc 4c001000.mailbox: stm32_ipcc_rx_irq: chan:0 rx
[ 115.757969] remoteproc remoteproc0: vq index 0 is interrupted
[Above logs are for STM32 platform. From your post, I can say you are using Xilinx Platform, and so above logs are not expected on linux side, as different drivers are used for Xilinx Platform.
For further confirmation, once zephyr firmware starts, please check following path on linux side for rpmsg devices:
ls /sys/bus/rpsmg/devices/*
You should see rpmsg devices created.
tnmysh
commented
2 months ago Check that before running the Zephyr firmware the kernel modules insmod rpmsg_client_sample.ko rpmsg_tty.ko rpmsg_char.ko rpmsg_ctrl.ko are inserted and running.
Have you inserted, rpmsg_ns.ko and virtio_rpmsg_bus.ko drivers?
olneumann
commented
2 months ago Check that before running the Zephyr firmware the kernel modules insmod rpmsg_client_sample.ko rpmsg_tty.ko rpmsg_char.ko rpmsg_ctrl.ko are inserted and running.
Have you inserted, rpmsg_ns.ko and virtio_rpmsg_bus.ko drivers?
Yes, they are inserted as well. I was also seeing them under the rpmsg/devices folder. The rpmsg_ctrl and rpmsg_ns.
Sadly I don't have access to the setup, but I will provide the traces and more logs from next week onwards.
tnmysh
commented
2 months ago One more debug point. Zephyr side mostly things look good. Could you paste linux side device-tree IPI nodes ?
If devices are created successfully, then I believe name service announcement is working, and so IPI is working too.
To verify, you do cat /proc/interrupts after starting remote processor. This will print all the interrupts occured in the system. Your IPI interrupt number should show increased counts.
tnmysh
commented
2 months ago I missed that linux device-tree was already posted. Your assesment is correct, it's IPI issue.
Linux device-tree
interrupts = <0 29 4>; // TODO: check if this is correct
xlnx,ipi-id = <7>; // TODO: check if this is correctZephyr device-tree:
rpu0_apu_mailbox: mailbox@ff990200 {
remote-ipi-id = <0>;So, zephyr fw is expecting APU ID to be 0, but APU ID is 7 instead.
Linux side device-tree zynqmp-ipi node with IPI ID 0 is introduced here: https://github.com/torvalds/linux/blob/e936e7d4a83b5ff6b7a685722f0ba348383af68c/arch/arm64/boot/dts/xilinx/zynqmp.dtsi#L139
and then for openamp additional nodes are added as following: https://github.com/OpenAMP/openamp-system-reference/blob/d57bae7346f2d6f0513d9eb8b22fda698d4af8e0/examples/linux/dts/xilinx/zynqmp-openamp.dtso#L61
I hope this will fix the issue.
olneumann
commented
1 month ago With the right overlay it works.
zynqmp-ipi {
#address-cells = <2>;
#size-cells = <2>;
ranges;
ipi_mailbox_rpu0: mailbox@ff990040 {
reg = <0x00 0xff990040 0x00 0x20>,
<0x00 0xff990060 0x00 0x20>,
<0x00 0xff990200 0x00 0x20>,
<0x00 0xff990220 0x00 0x20>;
reg-names = "local_request_region",
"local_response_region",
"remote_request_region",
"remote_response_region";
#mbox-cells = <0x01>;
xlnx,ipi-id = <0x01>;
};
ipi_mailbox_rpu1: mailbox@ff990080 {
reg = <0x00 0xff990080 0x00 0x20>,
<0x00 0xff9900a0 0x00 0x20>,
<0x00 0xff990400 0x00 0x20>,
<0x00 0xff990420 0x00 0x20>;
reg-names = "local_request_region",
"local_response_region",
"remote_request_region",
"remote_response_region";
#mbox-cells = <0x01>;
xlnx,ipi-id = <0x02>;
};
};Sorry I couldn't reply earlier. Returning this week from long leave. I am glad it worked!
Hi all,
I was trying to get the rpmsg-multi-service running with the PetaLinux Build Process and the Kria KR260 Evaluation Board (should be very similar to the KV260). But in the end I still get no RX response or the message that a virtio channel was created as it should be like in the following dump. I think it is due to the wrong IPM or IPI settings that I am not very familiar with.
Steps to Reproduce
PetaLinux Setup
1) I use the PetaLinux 2024.1 Tools and create a new project, add the Kria KR260 Starter Kit BSP and add the following
system-user.dtsiin theplnx_ipc/project-spec/meta-user/recipes-bsp/device-treefolder:system-user.dtsi
``` /include/ "system-conf.dtsi" / { reserved-memory { #address-cells = <0x02>; // determines the number of cells used to represent a register address in a reg property #size-cells = <0x02>; // determines the number of cells used to represent a register size in a reg property ranges; /* * xlnx_r5_remoteproc.c * * - vdev0buffer has to be used https://github.com/Xilinx/linux-xlnx/blob/d18bb880a47087bb8e49b17780c8265978cecaa5/drivers/remoteproc/xlnx_r5_remoteproc.c#L466C1-L467C47 * - vdev0ringx not specified in the driver * */ rproc_0_fw_image: memory@3ed00000 { no-map; reg = <0x0 0x3ed00000 0x0 0x40000>; }; rpu0vdev0vring0: vdev0vring0@3ed40000 { no-map; reg = <0x0 0x3ed40000 0x0 0x4000>; }; rpu0vdev0vring1: vdev0vring1@3ed44000 { no-map; reg = <0x0 0x3ed44000 0x0 0x4000>; }; rpu0vdev0buffer: vdev0buffer@3ed48000 { no-map; compatible = "shared-dma-pool"; reg = <0x0 0x3ed48000 0x0 0x100000>; }; rproc_1_fw_image: memory@3ef00000 { no-map; reg = <0x0 0x3ef00000 0x0 0x40000>; }; rpu1vdev0vring0: vdev0vring0@3ef40000 { no-map; reg = <0x0 0x3ef40000 0x0 0x4000>; }; rpu1vdev0vring1: vdev0vring1@3ef44000 { no-map; reg = <0x0 0x3ef44000 0x0 0x4000>; }; rpu1vdev0buffer: vdev0buffer@3ef48000 { no-map; compatible = "shared-dma-pool"; reg = <0x0 0x3ef48000 0x0 0x100000>; }; }; /* * IPI mailbox node * see Documentation/devicetree/bindings/mailbox/xlnx,zynqmp-ipi-mailbox.yaml */ zynqmp-mailbox { compatible = "xlnx,zynqmp-ipi-mailbox"; interrupt-parent = <&gic>; interrupts = <0 29 4>; // TODO: check if this is correct xlnx,ipi-id = <7>; // TODO: check if this is correct #address-cells = <1>; #size-cells = <1>; ranges; bootph-all; /* * Note: The software might reassign the interrupt channels and message buffers except * for the PMU interrupts. */ rpu0_apu_mailbox: mailbox@ff990040 { reg = <0xff990040 0x20>, // switched local and remote based on zephyr dts <0xff990060 0x20>, <0xff990200 0x20>, <0xff990220 0x20>; reg-names = "local_request_region", "local_response_region", "remote_request_region", "remote_response_region"; #mbox-cells = <1>; xlnx,ipi-id = <1>; bootph-all; }; rpu1_apu_mailbox: mailbox@ff990080 { reg = <0xff990080 0x20>, // TRM, Table 13-3: channel 3 <0xff9900a0 0x20>, <0xff990400 0x20>, <0xff990420 0x20>; reg-names = "local_request_region", "local_response_region", "remote_request_region", "remote_response_region"; #mbox-cells = <1>; xlnx,ipi-id = <2>; bootph-all; }; }; /* * RPU remoteproc node * see Documentation/devicetree/bindings/remoteproc/xlnx,zynamp-r5fss.yaml */ zynqmp-rpu { compatible = "xlnx,zynqmp-r5fss"; xlnx,cluster-mode = <0>; // split mode #address-cells = <2>; #size-cells = <2>; r5f-0 { compatible = "xlnx,zynqmp-r5f"; power-domains = <&zynqmp_firmware 0x7>; memory-region = <&rproc_0_fw_image>, <&rpu0vdev0buffer>, <&rpu0vdev0vring0>, <&rpu0vdev0vring1>; mboxes = <&rpu0_apu_mailbox 0>, <&rpu0_apu_mailbox 1>; mbox-names = "tx", "rx"; }; r5f-1 { compatible = "xlnx,zynqmp-r5f"; power-domains = <&zynqmp_firmware 0x8>; memory-region = <&rproc_1_fw_image>, <&rpu1vdev0buffer>, <&rpu1vdev0vring0>, <&rpu1vdev0vring1>; mboxes = <&rpu1_apu_mailbox 0>, <&rpu1_apu_mailbox 1>; mbox-names = "tx", "rx"; }; }; }; ```2) Patching the
virtio_rpmsg_bus.cfile of the xlnx_rebase_v6.6_LTS_2024.1_update branch. Allowing the xlnx_r5_remoteproc.c to be loaded as kernel module instead of the zynqmp_r5_remoteproc.c one. See this patch. 3) Selecting for the rootfs the packagegroup-petalinux-openamp.bb. 4) Build and flash. 5) Build and install the rpmsg-utils tools as root. 6) Check that before running the Zephyr firmware the kernel modulesinsmod rpmsg_client_sample.ko rpmsg_tty.ko rpmsg_char.ko rpmsg_ctrl.koare inserted and running.Until here I do not get any errors in dmesg.
Zephyr Firmware Setup
1) Using the provide main_remote.c of the rpmsg-multi-services example in a dedicated zephyr project firmware_zynqmp_ipc (branch: backport-zephyr-3.6.0). 2) The custom board is just a mirror of the KV260 board and goes with the following board overlay for the shm directives:
kr260_r5.overlay
``` / { chosen { zephyr,ipc = &rpu0_apu_mailbox; zephyr,ipc_shm = &rpu0_ipc_shm; zephyr,sram = &tcm; /delete-property/ zephyr,flash; }; soc { tcm: memory@0 { compatible = "mmio-sram"; reg = <0 0x40000>; }; }; reserved-memory { compatible = "reserved-memory"; #address-cells = <1>; #size-cells = <1>; status = "okay"; ranges; rpu0_ipc_shm: memory@3ed40000 { reg = <0x3ed40000 0x100000>; }; }; }; &rpu0_ipi { status = "okay"; }; &rpu0_apu_mailbox { status = "okay"; }; /delete-node/ &uart0; /delete-node/ &gem0; /delete-node/ &gem1; /delete-node/ &gem2; /delete-node/ &gem3; ```3) Loading the .elf to the remoteproc0 and starting it. Terminal gives me the output:
4) For reference the zynqmp_rpu.dtsi link which is the baseline for the board overlay.
Further Thoughts
So the firmware gets loaded properly and also is able to create this endpoint based on the output. How can I now proceed and debug the situation? I see in the dmesg output that when i create for example a tty endpoint and send something a similar output to this:
but then I didn't get anything further indicating that the remoteproc does something, i.e. from the example:
Any help here? Thank you.