agherzan
commented
1 year ago Could we have missed some upstream kernel fixes? Can you check the versions against the RaspbianOS one?
Closed bigbearishappy closed 12 months ago
agherzan
commented
1 year ago Could we have missed some upstream kernel fixes? Can you check the versions against the RaspbianOS one?
bigbearishappy
commented
1 year ago If we missed some upstream fixes, the pwm should not work well with changing the /boot/config.txt. I test with both kernel version 5.15 and 6.1 on raspbian OS. Both two of them work fine with pwm.
bigbearishappy
commented
12 months ago The fix code is merged to branch kirkstone.Closing the issue.
djr-spectrummfg
commented
9 months ago Oddly, this change breaks my system. I am using (only) a DSI display as well as psplash and weston on a CM4 with dtoverlay=vc4-fkms-v3d. I was previously running using the kirkstone branch commit immediately prior (43683cb14b6afc144619335b3a2353b70762ff3e). After this change I get very frequent kernel stall messages like below:
[ 11.790508] audit: type=1327 audit(1651167753.343:11): proctitle="(weston)"
[ 11.880753] Unable to handle kernel execute from non-executable memory at virtual address ffffff800614d78c
[ 11.890686] Mem abort info:
[ 11.893535] ESR = 0x8600000f
[ 11.896893] EC = 0x21: IABT (current EL), IL = 32 bits
[ 11.902458] SET = 0, FnV = 0
[ 11.905568] EA = 0, S1PTW = 0
[ 11.908937] FSC = 0x0f: level 3 permission fault
[ 11.913950] swapper pgtable: 4k pages, 39-bit VAs, pgdp=000000000120a000
[ 11.920816] [ffffff800614d78c] pgd=180000003b3f9003, p4d=180000003b3f9003, pud=180000003b3f9003, pmd=180000003b3cf003, pte=006800000614df07
[ 11.933621] Internal error: Oops: 8600000f [#1] PREEMPT SMP
[ 11.939288] Modules linked in: cfg80211 cdc_ether rfkill joydev v3d gpu_sched bcm2835_v4l2(C) dwc2 bcm2835_codec(C) raspberrypi_ts videobuf2_vmalloc raspberrypi_hwmon roles bcm2835_isp(C) v4l2_mem2mem bcm2835_mmal_vchiq(C) videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common videodev vc_sm_cma(C) vc4 mc cec drm_kms_helper syscopyarea rpivid_mem sysfillrect sysimgblt fb_sys_fops nvmem_rmem uio_pdrv_genirq uio sch_fq_codel kdmxio(O) i2c_dev i2c_bcm2708 drm fuse drm_panel_orientation_quirks backlight ipv6
[ 11.939494] CPU: 1 PID: 126 Comm: psplash Tainted: G C O 5.15.34-v8 #1
[ 11.993896] Hardware name: Raspberry Pi Compute Module 4 Rev 1.0 (DT)
[ 11.993904] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 11.993917] pc : 0xffffff800614d78c
[ 11.993928] lr : fb_release+0x44/0x74
[ 12.014754] sp : ffffffc0085b3c90
[ 12.014760] x29: ffffffc0085b3c90 x28: ffffff8003b28000 x27: 0000000000000000
[ 12.014782] x26: 0000000000000000 x25: 0000000000000000 x24: ffffff8003b28600
[ 12.032621] x23: ffffff800267c7a0 x22: ffffff80020a5180 x21: ffffff8002640628
[ 12.039874] x20: ffffff80025c1010 x19: ffffff80025c1000 x18: 0000000000000000
[ 12.047124] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[ 12.054374] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
[ 12.061623] x11: 0000000000000000 x10: 0000000000000001 x9 : ffffffe15feeb03c
[ 12.068873] x8 : 0000000000000004 x7 : ffffff8003c904e8 x6 : 0000000000000000
[ 12.076123] x5 : ffffff8002640628 x4 : 0000000000000000 x3 : ffffff80025c1010
[ 12.083372] x2 : ffffff800614d78c x1 : 0000000000000001 x0 : ffffff80025c1000
[ 12.090623] Call trace:
[ 12.093103] 0xffffff800614d78c
[ 12.096289] __fput+0x78/0x240
[ 12.099394] ____fput+0x1c/0x30
[ 12.102583] task_work_run+0x90/0x190
[ 12.106303] do_exit+0x34c/0xa10
[ 12.109581] do_group_exit+0x44/0xac
[ 12.113210] __wake_up_parent+0x0/0x3c
[ 12.117016] invoke_syscall+0x50/0x120
[ 12.120824] el0_svc_common.constprop.0+0x68/0x124
[ 12.125690] do_el0_svc+0x34/0x9c
[ 12.129055] el0_svc+0x2c/0x90
[ 12.132155] el0t_64_sync_handler+0xa4/0x130
[ 12.136487] el0t_64_sync+0x1a0/0x1a4
[ 12.140210] Code: 00000000 002a0f05 02100703 00000002 (0003100a)
[ 12.146395] ---[ end trace fac8972a0bf3b66f ]---
[ 12.151350] Fixing recursive fault but reboot is needed!OR:
[ 31.281311] rcu: INFO: rcu_preempt self-detected stall on CPU
[ 31.287166] rcu: 3-....: (5249 ticks this GP) idle=4c5/1/0x4000000000000002 softirq=1986/1986 fqs=1943
[ 31.296801] (t=5254 jiffies g=1297 q=1714)
[ 31.301047] Task dump for CPU 3:
[ 31.304318] task:psplash state:R running task stack: 0 pid: 129 ppid: 1 flags:0x0000000e
[ 31.314394] Call trace:
[ 31.316871] dump_backtrace+0x0/0x1b0
[ 31.320603] show_stack+0x24/0x30
[ 31.323971] sched_show_task+0x164/0x190
[ 31.327955] dump_cpu_task+0x50/0x60
[ 31.331584] rcu_dump_cpu_stacks+0xe0/0x160
[ 31.335832] rcu_sched_clock_irq+0xdc4/0xf30
[ 31.340166] update_process_times+0xb0/0x100
[ 31.344503] tick_sched_handle+0x3c/0x60
[ 31.348483] tick_sched_timer+0x84/0xf0
[ 31.352375] __hrtimer_run_queues+0x154/0x3a0
[ 31.356798] hrtimer_interrupt+0x100/0x260
[ 31.360958] arch_timer_handler_phys+0x40/0x50
[ 31.365469] handle_percpu_devid_irq+0xb0/0x244
[ 31.370070] handle_domain_irq+0xa4/0xf0
[ 31.374052] gic_handle_irq+0x58/0xe0
[ 31.377767] call_on_irq_stack+0x2c/0x5c
[ 31.381747] do_interrupt_handler+0x5c/0x70
[ 31.385992] el1_interrupt+0x30/0x80
[ 31.389620] el1h_64_irq_handler+0x18/0x24
[ 31.393775] el1h_64_irq+0x78/0x7c
[ 31.397224] queued_spin_lock_slowpath+0x22c/0x3c0
[ 31.402086] _raw_spin_lock+0x78/0x8c
[ 31.405805] __mutex_lock.constprop.0+0x9c/0x544
[ 31.410493] __mutex_lock_slowpath+0x1c/0x30
[ 31.414827] mutex_lock+0x54/0x60
[ 31.418191] fb_release+0x2c/0x74
[ 31.421557] __fput+0x78/0x240
[ 31.424659] ____fput+0x1c/0x30
[ 31.427846] task_work_run+0x90/0x190
[ 31.431565] do_exit+0x34c/0xa10
[ 31.434842] do_group_exit+0x44/0xac
[ 31.438470] __wake_up_parent+0x0/0x3c
[ 31.442275] invoke_syscall+0x50/0x120
[ 31.446083] el0_svc_common.constprop.0+0x68/0x124
[ 31.450947] do_el0_svc+0x34/0x9c
[ 31.454312] el0_svc+0x2c/0x90
[ 31.457411] el0t_64_sync_handler+0xa4/0x130
[ 31.461742] el0t_64_sync+0x1a0/0x1a4
[ 33.757390] cam-dummy-reg: disabling
[ 45.180847] IPv4: martian source 255.255.255.255 from 172.16.0.1, on dev eth0
[ 45.188154] ll header: 00000000: ff ff ff ff ff ff 2c 0b e9 28 59 8f 08 00
[ 94.481309] rcu: INFO: rcu_preempt self-detected stall on CPU
[ 94.487150] rcu: 3-....: (21004 ticks this GP) idle=4c5/1/0x4000000000000002 softirq=1986/1986 fqs=7970
[ 94.496869] (t=21054 jiffies g=1297 q=2805)
[ 94.501202] Task dump for CPU 3:
[ 94.504474] task:psplash state:R running task stack: 0 pid: 129 ppid: 1 flags:0x0000000e
[ 94.514547] Call trace:
[ 94.517024] dump_backtrace+0x0/0x1b0
[ 94.520750] show_stack+0x24/0x30
[ 94.524118] sched_show_task+0x164/0x190
[ 94.528103] dump_cpu_task+0x50/0x60
[ 94.531730] rcu_dump_cpu_stacks+0xe0/0x160
[ 94.535978] rcu_sched_clock_irq+0xdc4/0xf30
[ 94.540312] update_process_times+0xb0/0x100
[ 94.544650] tick_sched_handle+0x3c/0x60
[ 94.548630] tick_sched_timer+0x84/0xf0
[ 94.552521] __hrtimer_run_queues+0x154/0x3a0
[ 94.556946] hrtimer_interrupt+0x100/0x260
[ 94.561104] arch_timer_handler_phys+0x40/0x50
[ 94.565615] handle_percpu_devid_irq+0xb0/0x244
[ 94.570216] handle_domain_irq+0xa4/0xf0
[ 94.574195] gic_handle_irq+0x58/0xe0
[ 94.577910] call_on_irq_stack+0x2c/0x5c
[ 94.581891] do_interrupt_handler+0x5c/0x70
[ 94.586137] el1_interrupt+0x30/0x80
[ 94.589765] el1h_64_irq_handler+0x18/0x24
[ 94.593921] el1h_64_irq+0x78/0x7c
[ 94.597371] queued_spin_lock_slowpath+0x22c/0x3c0
[ 94.602232] _raw_spin_lock+0x78/0x8c
[ 94.605952] __mutex_lock.constprop.0+0x9c/0x544
[ 94.610639] __mutex_lock_slowpath+0x1c/0x30
[ 94.614973] mutex_lock+0x54/0x60
[ 94.618337] fb_release+0x2c/0x74
[ 94.621703] __fput+0x78/0x240
[ 94.624804] ____fput+0x1c/0x30
[ 94.627991] task_work_run+0x90/0x190
[ 94.631711] do_exit+0x34c/0xa10
[ 94.634988] do_group_exit+0x44/0xac
[ 94.638616] __wake_up_parent+0x0/0x3c
[ 94.642421] invoke_syscall+0x50/0x120
[ 94.646227] el0_svc_common.constprop.0+0x68/0x124
[ 94.651092] do_el0_svc+0x34/0x9c
[ 94.654457] el0_svc+0x2c/0x90
[ 94.657555] el0t_64_sync_handler+0xa4/0x130
[ 94.661887] el0t_64_sync+0x1a0/0x1a4(repeating every 60s or so)
The system also hangs on reboot:
...
[ OK ] Reached target System Reboot.
[ 187.580915] kauditd_printk_skb: 1 callbacks suppressed
[ 187.580933] audit: type=1334 audit(1651167929.171:17): prog-id=0 op=UNLOAD
[ 187.593219] audit: type=1334 audit(1651167929.171:18): prog-id=0 op=UNLOAD
[ 187.600258] audit: type=1334 audit(1651167929.191:19): prog-id=0 op=UNLOAD
[ 187.607295] audit: type=1334 audit(1651167929.191:20): prog-id=0 op=UNLOAD
[ 187.614396] audit: type=1334 audit(1651167929.199:21): prog-id=0 op=UNLOAD
[ 187.621506] audit: type=1334 audit(1651167929.199:22): prog-id=0 op=UNLOAD
[ 187.628561] audit: type=1334 audit(1651167929.203:23): prog-id=0 op=UNLOAD
[ 187.635592] audit: type=1334 audit(1651167929.203:24): prog-id=0 op=UNLOAD
[ 187.670304] systemd-shutdown[1]: Using hardware watchdog 'Broadcom BCM2835 Watchdog timer', version 0, device /dev/watchdog0
[ 187.681923] systemd-shutdown[1]: Failed to set timeout to 10min: Invalid argument
[ 187.720079] systemd-shutdown[1]: Syncing filesystems and block devices.
[ 187.735939] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
[ 187.767252] systemd-journald[131]: Received SIGTERM from PID 1 (systemd-shutdow).
[ 187.777605] audit: type=1335 audit(1651167929.367:25): pid=131 uid=0 auid=4294967295 tty=(none) ses=4294967295 comm="systemd-journal" exe="/lib/systemd/systemd-journald" nl-mcgrp=1 op=disconnect res=1
[ 197.766910] systemd-shutdown[1]: Waiting for process: 126 (psplash)
[ 197.766910] systemd-shutdown[1]: Waiting for process: 126 (psplash)
[ 277.773390] systemd-shutdown[1]: Sending SIGKILL to remaining processes...
[ 277.792251] systemd-shutdown[1]: Sending SIGKILL to PID 126 (psplash).
[ 287.802475] systemd-shutdown[1]: Waiting for process: 126 (psplash)Note psplash process cannot be manually killed even with kill -9. The kernel just gets stuck.
It seems clear that there is some underlying kernel bug that causes this as I have seen the same issue very occasionally with the old configuration, but for some reason changing to a modular build causes it to happen repeatably on every single boot and makes the system unusable.
kraj
commented
9 months ago I think it could be a race condition that module is not yet loaded and psplash is started now since its not built into kernel proper. So maybe add systemd-modules-load.service as dependency to psplash service might help.
djr-spectrummfg
commented
9 months ago I tried adding the below lines to the unit section of psplash-start.service but it did not make any difference:
After=systemd-modules-load.service
Requires=systemd-modules-load.serviceWhen I disable the psplash-start and psplash-systemd services and manually run psplash from the serial terminal (whether or not weston service is enabled/started) it works as expected with no stall. So it definitely seems like some sort of race condition...
Disabling the weston service but leaving the psplash services enabled (with the above dependency change) still results in the problem appearing so I don't think weston starting has anything to do with it.
Changing to a dtoverlay=vc4-kms-v3d based configuration (instead of dtoverlay=vc4-fkms-v3d) also doesn't fix the problem (and introduces a separate problem for me too).
djr-spectrummfg
commented
9 months ago Changing to a
dtoverlay=vc4-kms-v3dbased configuration (instead ofdtoverlay=vc4-fkms-v3d) also doesn't fix the problem (and introduces a separate problem for me too).
When I tested this, I forgot to remove my custom dt-blob.bin file that was necessary to get the vc4-fkms-v3d mode to work with the DSI screen. Removing this customised dt-blob (which added pin_defines for DISPLAY_I2C_PORT, DISPLAY_SDA, DISPLAY_SCL, and DISPLAY_DSI_PORT) of course breaks the vc4-fkms-v3d mode, but vc4-kms-v3d works and these kernel errors stop happening. So I can only guess that the firmware is doing something strange that isn't playing nicely with the linux drivers.
I think I am just going to go with the vc4-kms-v3d mode to avoid this problem, especially since I had seen the bug manifest occasionally before the change to building as modules.
Description I am trying to drive a LCD of ili9881d with CM4. And I use the customize dts file from here. I add it to my yocto(kirkstone) project successfully.But the lcd display nothing. After I connect it's pwm pin(GPIO13) to 3.3V. The lcd works fine.
Steps to reproduce the issue: 1.add new dts and relative kernel driver to yocto project. Based on this repo 2.flash the image and boot it 3.LCD shows nothing
Describe the results you received: I find that the PWM pin is in voltage 0V.After I connect it to 3.3V, the LCD works fine.
Describe the results you expected: The lcd should display the login console of yocto image.
Additional information you deem important (e.g. issue happens only occasionally): I tried to test the pwm function with adding
dtoverlay=pwm,pin=13,func=4to /boot/config.txt Then use the following command to control the pwm output:PWM works fine and I can get the voltage of 1.65V.
Additional details (revisions used, host distro, etc.): revision:
host distro:
One more thing: The customize dts file and kernel driver I used work fine with the raspbian OS.
I am wondering if I missed some configuration in yocto project. Looking forward to your reply~