search

Xilinx / systemctlm-cosim-demo

QEMU libsystemctlm-soc co-simulation demos.
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/862421112/Co-simulation
Other
131 stars 47 forks source link

Issue when reading IRQ in PS sent by PL #10

Open KRolander opened 4 years ago

KRolander commented 4 years ago

Dear all,

My aim is to write a Kernel Device Driver that allows to do operations when an IRQ is sent by the debug module.

In zynq_demo.cc the debug module's IRQ port is binded with the IRQ port of the zynq module : debug.irq(zynq.pl2ps_irq[0]);. I found that the IRQ signals are set into rp_wires_in.wires_in (rp_wires_in.wires_in[i]((pl2ps_irq[i])) members of the xilinx_zynq class.

After studying the Device Tree Source of remote ports of the zynq-7000 zynq-pl-remoteport.dtsi I found that the rp_wires_in@0 node corresponds to the rp_wires_in.wires_in[i] of the xilinw_zynq class. In the rp_wires_in@0 node there are the following interrupts : 

wires_in: rp_wires_in@0 {
            compatible = "remote-port-gpio";
            remote-ports = < &cosim_rp_0 9 >;
            num-gpios = < 16 >;
            /* QEMU has a bug in the interrupts-extended parsing,
             * so we need to use interrupt-parent for the moment.
             */
interrupts = < 
                                       0x0 29 0x4
                                       0x0 30 0x4
                       0x0 31 0x4
                       0x0 32 0x4
                       0x0 33 0x4
                       0x0 34 0x4
                       0x0 35 0x4
                       0x0 36 0x4
                       0x0 52 0x4
                       0x0 53 0x4
                       0x0 54 0x4
                       0x0 55 0x4
                       0x0 56 0x4
                       0x0 57 0x4
                       0x0 58 0x4
                       0x0 59 0x4
>;

I guess that the interruption 29 (0x0 29 0x4) corresponds to the first IRQ of the zynq module debug.irq(zynq.pl2ps_irq[0]).

In my Kernel Device Driver (test_irq) when I initialize the driver I use the request_irq function to subscribe to the IRQ 29 : 

// Device Name
#define DEV_NAME "test_irq"

// Interrupt Request number
#define IRQ_NO 29

//Interrupt handler for IRQ 29.
static irqreturn_t irq_handler(int irq, void *dev_id)
{
    printk(KERN_INFO "[test_irq] Shared IRQ: Interrupt Occurred");
    return IRQ_HANDLED;
}

int init_module(void)
{

int result,
result = request_irq(IRQ_NO, irq_handler, IRQF_SHARED, DEV_NAME, (void *)(irq_handler));
        if(result != 0){
            printk(KERN_INFO "[test_irq] my_device: cannot register IRQ : %d", IRQ_NO);
            free_irq(IRQ_NO,(void *)(irq_handler));
        }
        else
        {
            printk(KERN_INFO "[test_irq] Request for IRQ : %d is done", IRQ_NO);

        }
 ....
}

After loading the module isnmod test_irq.ko I have the follwong error : 

[   29.701116] [test_irq] Loading test_irq
[   29.701227] [test_irq] Loaded test_irq 503
[   29.701282] genirq: Flags mismatch irq 29. 00000084 (test_irq) vs. 00000004 (f8003000.dmac)
[   29.701448] [test_irq] my_device: cannot register IRQ : 29
[   29.701448] ------------[ cut here ]------------
[   29.701614] WARNING: CPU: 1 PID: 376 at kernel/irq/manage.c:1707 __free_irq+0xbc/0x37c
[   29.701725] Trying to free already-free IRQ 29
[   29.701780] Modules linked in: test_irq(O+) dev_benchmark_ps(O) set_cpufreq(O) dev_sha_512_IP(O) dev_sha_256_IP(O) dev_debug_IP(O) dev_PM_v2(O) dev_OPP_Release(O) dev_OPP_Init(O)
[   29.702057] CPU: 1 PID: 376 Comm: insmod Tainted: G           O      5.5.0-rc5-76821-g52a3cadbb252 #5
[   29.702222] Hardware name: Xilinx Zynq Platform
[   29.702333] [<c03129c8>] (unwind_backtrace) from [<c030cb58>] (show_stack+0x10/0x14)
[   29.702444] [<c030cb58>] (show_stack) from [<c0ed5e4c>] (dump_stack+0xc0/0xd4)
[   29.702610] [<c0ed5e4c>] (dump_stack) from [<c0347c38>] (__warn+0xd0/0xf8)
[   29.702720] [<c0347c38>] (__warn) from [<c0348018>] (warn_slowpath_fmt+0x98/0xbc)
[   29.702886] [<c0348018>] (warn_slowpath_fmt) from [<c039f5c4>] (__free_irq+0xbc/0x37c)
[   29.703052] [<c039f5c4>] (__free_irq) from [<c039f904>] (free_irq+0x38/0x9c)
[   29.703218] [<c039f904>] (free_irq) from [<bf03624c>] (init_module+0xb4/0xd4 [test_irq])
[   29.703384] [<bf03624c>] (init_module [test_irq]) from [<c0302e08>] (do_one_initcall+0x50/0x234)
[   29.703550] [<c0302e08>] (do_one_initcall) from [<c03dbc18>] (do_init_module+0x60/0x234)
[   29.703660] [<c03dbc18>] (do_init_module) from [<c03de048>] (load_module+0x21d4/0x24c8)
[   29.703826] [<c03de048>] (load_module) from [<c03de4a0>] (sys_init_module+0x164/0x1a4)
[   29.703992] [<c03de4a0>] (sys_init_module) from [<c0301000>] (ret_fast_syscall+0x0/0x54)
[   29.704103] Exception stack(0xd78f9fa8 to 0xd78f9ff0)
[   29.704213] 9fa0:                   0000187c beaffefa 000bb058 0000187c 000bb008 000b8c7c
[   29.704379] 9fc0: 0000187c beaffefa 000001b9 00000080 beaffefa 00000002 000b8cc8 00000000
[   29.704490] 9fe0: beaffc28 beaffc18 00022554 b6dfc990
[   29.704600] ---[ end trace 61d553a6b9deff67 ]---
[   29.704766] do_init_module: 'test_irq'->init suspiciously returned 503, it should follow 0/-E convention
[   29.704766] do_init_module: loading module anyway...
[   29.704932] CPU: 1 PID: 376 Comm: insmod Tainted: G        W  O      5.5.0-rc5-76821-g52a3cadbb252 #5
[   29.705098] Hardware name: Xilinx Zynq Platform
[   29.705209] [<c03129c8>] (unwind_backtrace) from [<c030cb58>] (show_stack+0x10/0x14)
[   29.705319] [<c030cb58>] (show_stack) from [<c0ed5e4c>] (dump_stack+0xc0/0xd4)
[   29.705485] [<c0ed5e4c>] (dump_stack) from [<c03dbdd4>] (do_init_module+0x21c/0x234)
[   29.705651] [<c03dbdd4>] (do_init_module) from [<c03de048>] (load_module+0x21d4/0x24c8)
[   29.705761] [<c03de048>] (load_module) from [<c03de4a0>] (sys_init_module+0x164/0x1a4)
[   29.705927] [<c03de4a0>] (sys_init_module) from [<c0301000>] (ret_fast_syscall+0x0/0x54)
[   29.706038] Exception stack(0xd78f9fa8 to 0xd78f9ff0)
[   29.706149] 9fa0:                   0000187c beaffefa 000bb058 0000187c 000bb008 000b8c7c
[   29.706314] 9fc0: 0000187c beaffefa 000001b9 00000080 beaffefa 00000002 000b8cc8 00000000
[   29.706480] 9fe0: beaffc28 beaffc18 00022554 b6dfc990

The most surprising line of this error is: [ 29.701282] genirq: Flags mismatch irq 29. 00000084 (test_irq) vs. 00000004 (f8003000.dmac) that tells us that the f8003000.dmac requiers the IRQ (an the IRQ is not shared), that is quiet strange because in the Device Tree Source in the dmac node the IRQ 29 is not declared: 


&dmac_s {
    #interrupt-cells = <1>;
    interrupt-map-mask = <0 0 0 0>;
    interrupt-map = <0 0 0 &rp_cosim_intr_pstopl 0 28 4>, <0 0 0 &intc 0 13 4>,
                    <0 0 0 &rp_cosim_intr_pstopl 0 20 4>, <0 0 0 &intc 0 14 4>,
                    <0 0 0 &rp_cosim_intr_pstopl 0 21 4>, <0 0 0 &intc 0 15 4>,
                    <0 0 0 &rp_cosim_intr_pstopl 0 22 4>, <0 0 0 &intc 0 16 4>,
                    <0 0 0 &rp_cosim_intr_pstopl 0 23 4>, <0 0 0 &intc 0 17 4>,
                    <0 0 0 &rp_cosim_intr_pstopl 0 24 4>, <0 0 0 &intc 0 40 4>,
                    <0 0 0 &rp_cosim_intr_pstopl 0 25 4>, <0 0 0 &intc 0 41 4>,
                    <0 0 0 &rp_cosim_intr_pstopl 0 26 4>, <0 0 0 &intc 0 42 4>,
                    <0 0 0 &rp_cosim_intr_pstopl 0 27 4>, <0 0 0 &intc 0 43 4>;
};

After this error I declared a new IRQ (37) in the node rp_wires_in@0 so IRQ 37 corresponds to debug.irq(zynq.pl2ps_irq[8]) if I'm not wrong? After loading the module I have no error. In the /proc/interrupts I can see that my module test_irq has been subscribed to the IRQ 37 as we can see the log of cat /proc/interrupts:

           CPU0       CPU1       
 16:          0          0     GIC-0  43 Level     ttc_clockevent
 17:      19706      20537     GIC-0  29 Edge      twd
 18:          0          0     GIC-0  37 Level     arm-pmu
 19:          0          0     GIC-0  38 Level     arm-pmu
 20:         43          0     GIC-0  39 Level     f8007100.adc
 24:        352          0     GIC-0  82 Level     xuartps
 26:          9          0     GIC-0  54 Level     eth0
 27:          0          0     GIC-0  56 Level     mmc0
 28:          0          0     GIC-0  45 Level     f8003000.dmac
 29:          0          0     GIC-0  46 Level     f8003000.dmac
 30:          0          0     GIC-0  47 Level     f8003000.dmac
 31:          0          0     GIC-0  48 Level     f8003000.dmac
 32:          0          0     GIC-0  49 Level     f8003000.dmac
 33:          0          0     GIC-0  72 Level     f8003000.dmac
 34:          0          0     GIC-0  73 Level     f8003000.dmac
 35:          0          0     GIC-0  74 Level     f8003000.dmac
 36:          0          0     GIC-0  75 Level     f8003000.dmac
 37:          0          0     GIC-0  40 Level     test_irq
IPI0:          0          0  CPU wakeup interrupts
IPI1:          0          0  Timer broadcast interrupts
IPI2:       2362       3824  Rescheduling interrupts
IPI3:         57         49  Function call interrupts
IPI4:          0          0  CPU stop interrupts
IPI5:       2144       5829  IRQ work interrupts
IPI6:          0          0  completion interrupts
Err:          0

The device test_irq is subscribed to IRQ 37, when the debug module writes on its irq port (irq.write(1);) the device irq_handler function should be called to print the message: [test_irq] Shared IRQ: Interrupt Occurred, but the irq_handler function is not called.

Do you have any idea, how to solve this problem ? Maybe I should use offsets between the IRQs of dmac_s and rp_wires_in ? Or is it an issue of QEMU ?

Thank You, Best Regards

Roland

edgarigl commented 3 years ago

Hi Roland,

Sorry, I must have missed this issue when you filed it. Didn't see it until now...

I'm a little puzzled too. According to the Zynq TRM, IRQ 61 belongs to the PL (61 - 32 = 29).

Looking at the upstream dts for zynq-7000, dmac interrupts look like this: interrupt-names = "abort", "dma0", "dma1", "dma2", "dma3", "dma4", "dma5", "dma6", "dma7"; interrupts = <0 13 4>, <0 14 4>, <0 15 4>, <0 16 4>, <0 17 4>, <0 40 4>, <0 41 4>, <0 42 4>, <0 43 4>;

Which seems OK...

Did you find a solution for this already?

Thanks, Edgar

KRolander commented 3 years ago

Hi Edgard, no problem, fortunatly I found a solution and I found that IRQ 61 belongs to the PL (61 - 32 = 29) as well thank you.

So basically when we want to write a device driver subscribing on an IRQ of the Zynq in the Co-Simulation let's do in this way:

First of all we need an IRQ handler:

static irqreturn_t irq_handler(int irq, void *dev_id)
{
    wait_queue_flag = 1;
    wake_up_interruptible(&wait_queue_flag);
    return IRQ_HANDLED;
}

After we have to map the IRQ, so in the device init_module we need something like this:

int init_module(void)
{

        struct device_node *np;

    np = of_find_node_by_name(NULL, "rp_wires_in");

        int irq = irq_of_parse_and_map(np, 0);

        printk("====>>> IRQ num is : %d", irq);

        result = request_irq(irq, (irq_handler_t) irq_handler, 0x81, DEV_NAME, (void *)(irq_handler));

        if (result != 0)
        {
            printk("[test_irq] IRQ %d cannot be registered", irq);
            free_irq(irq, (void *)(irq_handler0));
        }
        else
        {
            printk("[device] IRQ %d is done", irq);
        }

}

Remarque: rp_wires_in is the node name in the DTS containing the IRQs and the PS-PL remote-ports to the.

0x81 in request_irq(irq, (irq_handler_t) irq_handler, 0x81, DEV_NAME, (void *)(irq_handler)); it's equal to say that the IRQ is a shared edge one, you can also use this flag: IRQ_SHARED_EDGE instead of using the numerical value.

In irq_of_parse_and_map(np, 0) => 0 refers to your_systemc_module.irq(zynq.pl2ps_irq[0]); for zynq.pl2ps_irq[1] you need irq_of_parse_and_map(np, 1) and so on...

So the IRQs in the DTS correspond to the SystemC ports in this way: 

interrupts = <
                       0x0 29 0x4        ========>   zynq.pl2ps_irq[0]
                       0x0 30 0x4        ========>   zynq.pl2ps_irq[1]
                       0x0 31 0x4        ========>   zynq.pl2ps_irq[2]
                       0x0 32 0x4        ========>   zynq.pl2ps_irq[3]
                       0x0 33 0x4        ========>   zynq.pl2ps_irq[4]
                       0x0 34 0x4        ========>   zynq.pl2ps_irq[5]
                       0x0 35 0x4        ========>   zynq.pl2ps_irq[6]
                       0x0 36 0x4        ========>   zynq.pl2ps_irq[7]

                       0x0 52 0x4        ========>   zynq.pl2ps_irq[8]
                       0x0 53 0x4        ========>   zynq.pl2ps_irq[9]
                       0x0 54 0x4        ========>   zynq.pl2ps_irq[10]
                       0x0 55 0x4        ========>   zynq.pl2ps_irq[11]
                       0x0 56 0x4        ========>   zynq.pl2ps_irq[12]
                       0x0 57 0x4        ========>   zynq.pl2ps_irq[13]
                       0x0 58 0x4        ========>   zynq.pl2ps_irq[14]
                       0x0 59 0x4        ========>   zynq.pl2ps_irq[15]
                     >;

So basically everithing work fine, I could generate IRQs in PL and subscribe on them by the Device Driver on the PS side :)

Best Regards Roland