Closed comdata closed 6 years ago
P33M
commented
6 years ago The top of the stacktrace is cut off, unfortunately.
Can you add framebuffer_height=2160 to /boot/config.txt, or alternatively enable_uart=1 and record the serial console output by dumping it on another device?
zerwes
commented
6 years ago same issue here:
[15086.999699] Unable to handle kernel NULL pointer dereference at virtual address 0000005c
[15087.003261] pgd = b6160000
[15087.005066] [0000005c] *pgd=36fc7835, *pte=00000000, *ppte=00000000
[15087.007027] Internal error: Oops: 17 [#1] SMP ARM
[15087.008981] Modules linked in: dm_mod dax cmac bnep hci_uart bluetooth ecdh_generic brcmfmac brcmutil cfg80211 rfkill snd_bcm2835(C) snd_pcm snd_timer snd fixed uio_pdrv_genirq uio sg ip_tables x_tables ipv6
[15087.015544] CPU: 0 PID: 5797 Comm: minidlnad Tainted: G C 4.14.30-v7+ #1102
[15087.020145] Hardware name: BCM2835
[15087.022514] task: b6195a00 task.stack: b5e90000
[15087.024938] PC is at tcp_push+0x64/0x148
[15087.027330] LR is at 0xe2515185
[15087.029717] pc : [<806ce3a4>] lr : [<e2515185>] psr: 60000013
[15087.032222] sp : b5e91ba0 ip : b633523c fp : b5e91bc4
[15087.034723] r10: b5f36c00 r9 : ffffffe0 r8 : b633523c
[15087.037201] r7 : 00008000 r6 : 000005a8 r5 : 00000000 r4 : b6335140
[15087.039733] r3 : 00000000 r2 : 1daf2e7a r1 : 000005a8 r0 : e2515185
[15087.042239] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[15087.044839] Control: 10c5383d Table: 3616006a DAC: 00000055
[15087.047375] Process minidlnad (pid: 5797, stack limit = 0xb5e90210)
[15087.049882] Stack: (0xb5e91ba0 to 0xb5e92000)
[15087.052410] 1ba0: 000005a8 806d1930 00000cba b5e91c98 b6335140 00000cba b5e91c5c b5e91bc8
[15087.057517] 1bc0: 806d1950 806ce34c 000005a8 801236e0 b5e91c10 80c046cc 80c069c4 80d0a08c
[15087.062692] 1be0: 80c049c0 b5e90008 00000000 804954bc 00000000 00000000 00000001 00000000
[15087.067884] 1c00: b5e91c84 b5e91c10 b5e91ca0 b6335290 000005a8 00028000 b5e91c90 000005a8
[15087.073241] 1c20: 7fffffff bb3941c0 b5e91c98 00000000 bd3907fc b6335140 bb3941c0 b5e91c98
[15087.078932] 1c40: 00000000 bd390bec 00000001 05253dce b5e91c84 b5e91c60 80657050 806d1654
[15087.084971] 1c60: 00001000 8014f388 b5e91c9c 00452bec 00001000 00000000 b5e91ce4 b5e91c88
[15087.091240] 1c80: 8065b568 80656ffc 00001000 00000000 9ebe3000 00001000 00000000 00000000
[15087.097819] 1ca0: 00000003 00000cba 00000346 b5e91c90 00000001 00000000 00000000 00000000
[15087.104560] 1cc0: 00028000 00000000 b6335140 bd390bec 00000000 00000000 b5e91d14 b5e91ce8
[15087.111456] 1ce0: 806d1584 8065b4ac 00028000 00028000 b5e91d14 b5e91d00 b6335140 00028000
[15087.118429] 1d00: 00001000 00000000 b5e91d44 b5e91d18 806d1630 806d1520 00028000 00000000
[15087.124261] 1d20: 5abd0f1a b6335140 bb3941c0 00000000 bc9f6c00 802bdb54 b5e91d7c b5e91d48
[15087.127758] 1d40: 806fef58 806d15f0 00028000 0e538128 b5e91d8c b5e91d60 802a6ff4 80187ab0
[15087.131267] 1d60: 00028000 bd390bec 00000000 bc9f6c00 b5e91d94 b5e91d80 80657bac 806fef08
[15087.134776] 1d80: 00028000 b5e91d90 b5e91dac b5e91d98 80657c04 80657b84 00028000 0e538128
[15087.138284] 1da0: b5e91dd4 b5e91db0 802bdbc4 80657bd4 b5e91db8 00028000 0000c000 00000000
[15087.141791] 1dc0: b6a11320 b5e91e00 b5e91dfc b5e91dd8 802bee10 802bdb60 bc9f6c00 8084da00
[15087.145286] 1de0: 00000000 00000000 00010000 0bb60000 b5e91e44 b5e91e00 802bf5d0 802bed08
[15087.148791] 1e00: 00004000 00001000 00000004 bc224600 0000c000 00000000 00000000 0000c000
[15087.152297] 1e20: 00000000 00000000 00000004 8084da00 0bb60000 00000000 b5e91e5c b5e91e48
[15087.155805] 1e40: 802bf618 802bf57c 00000004 802bdb54 b5e91e7c b5e91e60 802bdb4c 802bf5f4
[15087.159307] 1e60: 00000004 0bb60000 bc9f6c00 b5e91ed0 b5e91ecc b5e91e80 802be3f4 802bdb0c
[15087.162807] 1e80: 00000000 80000007 00000000 802bdb00 00000000 b5d84c00 0bb70000 00000000
[15087.166313] 1ea0: 8078a67c 00000000 00000000 b5e91f38 b5d84c00 bc224600 bc224600 10db3dce
[15087.169819] 1ec0: b5e91f14 b5e91ed0 802be618 802be314 00010000 10db3dce 00000004 bc224600
[15087.173324] 1ee0: 0bb60000 00000000 b5e91f40 00000000 00000000 00000000 b5d84c00 b5d84c00
[15087.176825] 1f00: 00000000 b5e91f80 b5e91f74 b5e91f18 80289fd4 802be588 10db3dce 00000000
[15087.180332] 1f20: 00000020 00000000 bb3941e0 b5d84c00 00000000 00000000 00000000 00000000
[15087.183843] 1f40: 00000000 00000000 0000000b 7e8db910 10db3dce 00000000 00000000 80108224
[15087.187356] 1f60: b5e90000 00000000 b5e91fa4 b5e91f78 8028b5d0 80289e24 fffff000 000001ff
[15087.190867] 1f80: 00000000 00000000 7ffffffe 00000000 00000000 000000ef 00000000 b5e91fa8
[15087.194367] 1fa0: 80108080 8028b554 7ffffffe 00000000 0000000a 0000000b 7e8db910 10db3dce
[15087.197876] 1fc0: 7ffffffe 00000000 00000000 000000ef 020b0d20 00000000 10db3dcd 00000000
[15087.201386] 1fe0: 769e6340 7e8db8ec 00018380 769e634c 60000010 0000000a d19f3bd3 74d33abf
[15087.204905] [<806ce3a4>] (tcp_push) from [<806d1950>] (tcp_sendmsg_locked+0x308/0xdfc)
[15087.208421] [<806d1950>] (tcp_sendmsg_locked) from [<80657050>] (kernel_sendmsg_locked+0x60/0x78)
[15087.211946] [<80657050>] (kernel_sendmsg_locked) from [<8065b568>] (sock_no_sendpage_locked+0xc8/0xd0)
[15087.215494] [<8065b568>] (sock_no_sendpage_locked) from [<806d1584>] (tcp_sendpage_locked+0x70/0xd0)
[15087.219050] [<806d1584>] (tcp_sendpage_locked) from [<806d1630>] (tcp_sendpage+0x4c/0x64)
[15087.222604] [<806d1630>] (tcp_sendpage) from [<806fef58>] (inet_sendpage+0x5c/0x114)
[15087.226151] [<806fef58>] (inet_sendpage) from [<80657bac>] (kernel_sendpage+0x34/0x50)
[15087.229706] [<80657bac>] (kernel_sendpage) from [<80657c04>] (sock_sendpage+0x3c/0x44)
[15087.233277] [<80657c04>] (sock_sendpage) from [<802bdbc4>] (pipe_to_sendpage+0x70/0x94)
[15087.236858] [<802bdbc4>] (pipe_to_sendpage) from [<802bee10>] (__splice_from_pipe+0x114/0x1a8)
[15087.240452] [<802bee10>] (__splice_from_pipe) from [<802bf5d0>] (splice_from_pipe+0x60/0x78)
[15087.244036] [<802bf5d0>] (splice_from_pipe) from [<802bf618>] (generic_splice_sendpage+0x30/0x38)
[15087.247629] [<802bf618>] (generic_splice_sendpage) from [<802bdb4c>] (direct_splice_actor+0x4c/0x54)
[15087.251229] [<802bdb4c>] (direct_splice_actor) from [<802be3f4>] (splice_direct_to_actor+0xec/0x274)
[15087.254832] [<802be3f4>] (splice_direct_to_actor) from [<802be618>] (do_splice_direct+0x9c/0xc4)
[15087.258432] [<802be618>] (do_splice_direct) from [<80289fd4>] (do_sendfile+0x1bc/0x3e0)
[15087.262011] [<80289fd4>] (do_sendfile) from [<8028b5d0>] (SyS_sendfile64+0x88/0xf8)
[15087.265598] [<8028b5d0>] (SyS_sendfile64) from [<80108080>] (ret_fast_syscall+0x0/0x28)
[15087.269189] Code: e5840514 e3110001 e59b1004 158404a4 (e595205c)
[15087.271065] ---[ end trace 56186a5c056622b6 ]---
[21482.174013] Unable to handle kernel NULL pointer dereference at virtual address 0000005c
[21482.181053] pgd = b5d80000
[21482.184413] [0000005c] *pgd=36f59835, *pte=00000000, *ppte=00000000
[21482.187884] Internal error: Oops: 17 [#2] SMP ARM
[21482.191193] Modules linked in: dm_mod dax cmac bnep hci_uart bluetooth ecdh_generic brcmfmac brcmutil cfg80211 rfkill snd_bcm2835(C) snd_pcm snd_timer snd fixed uio_pdrv_genirq uio sg ip_tables x_tables ipv6
[21482.201216] CPU: 0 PID: 7839 Comm: minidlnad Tainted: G D C 4.14.30-v7+ #1102
[21482.207643] Hardware name: BCM2835
[21482.210745] task: b6b23c00 task.stack: bb898000
[21482.213799] PC is at tcp_push+0x64/0x148
[21482.216744] LR is at 0x3f129b82
[21482.219559] pc : [<806ce3a4>] lr : [<3f129b82>] psr: 60000013
[21482.222394] sp : bb899ba0 ip : b63345bc fp : bb899bc4
[21482.225158] r10: b5f36300 r9 : ffffffe0 r8 : b63345bc
[21482.227831] r7 : 00008000 r6 : 000005a8 r5 : 00000000 r4 : b63344c0
[21482.230467] r3 : 00000000 r2 : c0ede47d r1 : 000005a8 r0 : 3f129b82
[21482.233040] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[21482.235653] Control: 10c5383d Table: 35d8006a DAC: 00000055
[21482.238211] Process minidlnad (pid: 7839, stack limit = 0xbb898210)
[21482.240740] Stack: (0xbb899ba0 to 0xbb89a000)
[21482.243291] 9ba0: 000005a8 806d1930 00000be8 bb899c98 b63344c0 00000be8 bb899c5c bb899bc8
[21482.248443] 9bc0: 806d1950 806ce34c 000005a8 b6b23c00 bc2a0c40 bceefd40 80c069c4 80d0a08c
[21482.253554] 9be0: 80c049c0 bb898008 00000000 80b8cd40 00000000 00000000 00000001 00000000
[21482.256206] 9c00: 80c82430 80c03d68 bb899ca0 b6334610 000005a8 00028000 808024bc 000005a8
[21482.258915] 9c20: 7fffffff bceefd40 00000001 00000000 bb898000 b63344c0 bc44aa80 bb899c98
[21482.261791] 9c40: 00000000 bd382768 00000001 046b3d7d bb899c84 bb899c60 80657050 806d1654
[21482.264837] 9c60: 00001000 bd382744 bb899c84 00444768 00001000 00000000 bb899ce4 bb899c88
[21482.267994] 9c80: 8065b568 80656ffc 00001000 bb899c98 9e58a000 00001000 00000000 00000000
[21482.271313] 9ca0: 00000003 00000be8 00000418 bb899c90 00000001 00000000 00000000 00000000
[21482.274716] 9cc0: 00028000 00000000 b63344c0 bd382768 00000000 00000000 bb899d14 bb899ce8
[21482.278192] 9ce0: 806d1584 8065b4ac 00028000 00028000 bb899d14 bb899d00 b63344c0 00028000
[21482.281703] 9d00: 00001000 00000000 bb899d44 bb899d18 806d1630 806d1520 00028000 b6b23c00
[21482.285213] 9d20: bc2a0c40 b63344c0 bc44aa80 00000000 b638f000 802bdb54 bb899d7c bb899d48
[21482.288726] 9d40: 806fef58 806d15f0 00028000 80145970 bb899d8c bb899d60 802a6ff4 80187ab0
[21482.292239] 9d60: 00028000 bd382768 00000000 b638f000 bb899d94 bb899d80 80657bac 806fef08
[21482.295752] 9d80: 00028000 bb898000 bb899dac bb899d98 80657c04 80657b84 00028000 046b3d7d
[21482.299258] 9da0: bb899dd4 bb899db0 802bdbc4 80657bd4 bb899db8 00028000 00001000 00000000
[21482.302766] 9dc0: b8225618 bb899e00 bb899dfc bb899dd8 802bee10 802bdb60 b638f000 8084da00
[21482.306280] 9de0: 00000000 00000000 00010000 015f0000 bb899e44 bb899e00 802bf5d0 802bed08
[21482.309790] 9e00: 0000f000 00001000 00000004 b6396b40 00001000 00000000 00000000 00001000
[21482.313298] 9e20: 00000000 00000000 00000004 8084da00 015f0000 00000000 bb899e5c bb899e48
[21482.316795] 9e40: 802bf618 802bf57c 00000004 802bdb54 bb899e7c bb899e60 802bdb4c 802bf5f4
[21482.320302] 9e60: 00000004 015f0000 b638f000 bb899ed0 bb899ecc bb899e80 802be3f4 802bdb0c
[21482.323810] 9e80: 00000000 80000007 00000000 802bdb00 00000000 b5d3b240 01600000 00000000
[21482.327324] 9ea0: 8078a67c 00000000 00000000 bb899f38 b5d3b240 b6396b40 b6396b40 05ca3d7d
[21482.330837] 9ec0: bb899f14 bb899ed0 802be618 802be314 00010000 05ca3d7d 00000004 b6396b40
[21482.334341] 9ee0: 015f0000 00000000 bb899f40 00000000 00000000 00000000 b5d3b240 b5d3b240
[21482.337856] 9f00: 00000000 bb899f80 bb899f74 bb899f18 80289fd4 802be588 05ca3d7d 00000000
[21482.341370] 9f20: 00000020 00000000 bc44aaa0 b5d3b240 00000000 00000000 00000000 00000000
[21482.344891] 9f40: 00000000 00000000 0000000b 7e8db910 05ca3d7d 00000000 00000000 80108224
[21482.348407] 9f60: bb898000 00000000 bb899fa4 bb899f78 8028b5d0 80289e24 fffff000 000001ff
[21482.351921] 9f80: 00000000 00000000 7ffffffe 00000000 00000000 000000ef 00000000 bb899fa8
[21482.355444] 9fa0: 80108080 8028b554 7ffffffe 00000000 0000000a 0000000b 7e8db910 05ca3d7d
[21482.358963] 9fc0: 7ffffffe 00000000 00000000 000000ef 0200e0d8 00000000 05ca3d7c 00000000
[21482.362486] 9fe0: 769e6340 7e8db8ec 00018380 769e634c 60000010 0000000a 995aecfc 5ac341e5
[21482.366009] [<806ce3a4>] (tcp_push) from [<806d1950>] (tcp_sendmsg_locked+0x308/0xdfc)
[21482.369536] [<806d1950>] (tcp_sendmsg_locked) from [<80657050>] (kernel_sendmsg_locked+0x60/0x78)
[21482.373085] [<80657050>] (kernel_sendmsg_locked) from [<8065b568>] (sock_no_sendpage_locked+0xc8/0xd0)
[21482.376643] [<8065b568>] (sock_no_sendpage_locked) from [<806d1584>] (tcp_sendpage_locked+0x70/0xd0)
[21482.380208] [<806d1584>] (tcp_sendpage_locked) from [<806d1630>] (tcp_sendpage+0x4c/0x64)
[21482.383754] [<806d1630>] (tcp_sendpage) from [<806fef58>] (inet_sendpage+0x5c/0x114)
[21482.387312] [<806fef58>] (inet_sendpage) from [<80657bac>] (kernel_sendpage+0x34/0x50)
[21482.390872] [<80657bac>] (kernel_sendpage) from [<80657c04>] (sock_sendpage+0x3c/0x44)
[21482.394440] [<80657c04>] (sock_sendpage) from [<802bdbc4>] (pipe_to_sendpage+0x70/0x94)
[21482.398014] [<802bdbc4>] (pipe_to_sendpage) from [<802bee10>] (__splice_from_pipe+0x114/0x1a8)
[21482.401596] [<802bee10>] (__splice_from_pipe) from [<802bf5d0>] (splice_from_pipe+0x60/0x78)
[21482.405185] [<802bf5d0>] (splice_from_pipe) from [<802bf618>] (generic_splice_sendpage+0x30/0x38)
[21482.408779] [<802bf618>] (generic_splice_sendpage) from [<802bdb4c>] (direct_splice_actor+0x4c/0x54)
[21482.412375] [<802bdb4c>] (direct_splice_actor) from [<802be3f4>] (splice_direct_to_actor+0xec/0x274)
[21482.415963] [<802be3f4>] (splice_direct_to_actor) from [<802be618>] (do_splice_direct+0x9c/0xc4)
[21482.419553] [<802be618>] (do_splice_direct) from [<80289fd4>] (do_sendfile+0x1bc/0x3e0)
[21482.423140] [<80289fd4>] (do_sendfile) from [<8028b5d0>] (SyS_sendfile64+0x88/0xf8)
[21482.426727] [<8028b5d0>] (SyS_sendfile64) from [<80108080>] (ret_fast_syscall+0x0/0x28)
[21482.430314] Code: e5840514 e3110001 e59b1004 158404a4 (e595205c)
[21482.432206] ---[ end trace 56186a5c056622b7 ]---another one
Mar 29 10:48:02 rasp3 kernel: [36470.552725] Unable to handle kernel NULL pointer dereference at virtual address 0000005c
Mar 29 10:48:02 rasp3 kernel: [36470.556421] pgd = 8319c000
Mar 29 10:48:02 rasp3 kernel: [36470.558234] [0000005c] *pgd=031bf835, *pte=00000000, *ppte=00000000
Mar 29 10:48:02 rasp3 kernel: [36470.560229] Internal error: Oops: 17 [#1] SMP ARM
Mar 29 10:48:02 rasp3 kernel: [36470.562234] Modules linked in: dm_mod dax cmac bnep hci_uart bluetooth ecdh_generic sg brcmfmac brcmutil cfg80211 rfkill snd_bcm2835(C) snd_pcm snd_timer snd fixed uio_pdrv_genirq uio ip_tables x_tables ipv6
Mar 29 10:48:02 rasp3 kernel: [36470.568826] CPU: 0 PID: 15212 Comm: minidlnad Tainted: G C 4.14.30-v7+ #1102
Mar 29 10:48:02 rasp3 kernel: [36470.573498] Hardware name: BCM2835
Mar 29 10:48:02 rasp3 kernel: [36470.575855] task: b9cc9e00 task.stack: 8339a000
Mar 29 10:48:02 rasp3 kernel: [36470.578266] PC is at tcp_push+0x64/0x148
Mar 29 10:48:02 rasp3 kernel: [36470.580663] LR is at 0x9267d752
Mar 29 10:48:02 rasp3 kernel: [36470.583056] pc : [<806ce3a4>] lr : [<9267d752>] psr: 60000013
Mar 29 10:48:02 rasp3 kernel: [36470.585568] sp : 8339bba0 ip : b9ff523c fp : 8339bbc4
Mar 29 10:48:02 rasp3 kernel: [36470.588073] r10: 83219800 r9 : ffffffe0 r8 : b9ff523c
Mar 29 10:48:02 rasp3 kernel: [36470.590550] r7 : 00008000 r6 : 000005a8 r5 : 00000000 r4 : b9ff5140
Mar 29 10:48:02 rasp3 kernel: [36470.593080] r3 : 00000000 r2 : 6d98a8ad r1 : 000005a8 r0 : 9267d752
Mar 29 10:48:02 rasp3 kernel: [36470.595592] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Mar 29 10:48:02 rasp3 kernel: [36470.598196] Control: 10c5383d Table: 0319c06a DAC: 00000055
Mar 29 10:48:02 rasp3 kernel: [36470.600740] Process minidlnad (pid: 15212, stack limit = 0x8339a210)
Mar 29 10:48:02 rasp3 kernel: [36470.603264] Stack: (0x8339bba0 to 0x8339c000)
Mar 29 10:48:02 rasp3 kernel: [36470.605810] bba0: 000005a8 806d1930 00000320 8339bc98 b9ff5140 00000320 8339bc5c 8339bbc8
Mar 29 10:48:02 rasp3 kernel: [36470.610949] bbc0: 806d1950 806ce34c 000005a8 8339bbd8 8014fbf0 80776f80 80c069c4 80d0a08c
Mar 29 10:48:02 rasp3 kernel: [36470.616156] bbe0: 80c049c0 8339a008 00000000 8339bbf8 00000000 00000000 00000000 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.621382] bc00: 8019c224 806256c8 8339bca0 b9ff5290 000005a8 00028000 8d4fdf3b 000005a8
Mar 29 10:48:02 rasp3 kernel: [36470.626770] bc20: 7fffffff 806e073c 83340780 00000000 8339bcb4 b9ff5140 bb3a3dc0 8339bc98
Mar 29 10:48:02 rasp3 kernel: [36470.632494] bc40: 00000000 bd115e4c 00000001 09763a61 8339bc84 8339bc60 80657050 806d1654
Mar 29 10:48:02 rasp3 kernel: [36470.638558] bc60: 00001000 80c05e98 80c02d00 001d7e4c 00001000 00000000 8339bce4 8339bc88
Mar 29 10:48:02 rasp3 kernel: [36470.644856] bc80: 8065b568 80656ffc 00001000 b9ff5140 8d1bb000 00001000 00000000 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.651468] bca0: 00000003 00000320 00000ce0 8339bc90 00000001 00000000 00000000 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.658239] bcc0: 00028000 00000000 b9ff5140 bd115e4c 00000000 00000000 8339bd14 8339bce8
Mar 29 10:48:02 rasp3 kernel: [36470.665142] bce0: 806d1584 8065b4ac 00028000 00028000 8339bd14 8339bd00 b9ff5140 00028000
Mar 29 10:48:02 rasp3 kernel: [36470.672125] bd00: 00001000 00000000 8339bd44 8339bd18 806d1630 806d1520 00028000 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.679106] bd20: 5abca841 b9ff5140 bb3a3dc0 00000000 9b118d80 802bdb54 8339bd7c 8339bd48
Mar 29 10:48:02 rasp3 kernel: [36470.686090] bd40: 806fef58 806d15f0 00028000 2f3fee21 8339bd8c 8339bd60 802a6ff4 80187ab0
Mar 29 10:48:02 rasp3 kernel: [36470.692817] bd60: 00028000 bd115e4c 00000000 9b118d80 8339bd94 8339bd80 80657bac 806fef08
Mar 29 10:48:02 rasp3 kernel: [36470.696335] bd80: 00028000 8339bd90 8339bdac 8339bd98 80657c04 80657b84 00028000 2f3fee21
Mar 29 10:48:02 rasp3 kernel: [36470.699850] bda0: 8339bdd4 8339bdb0 802bdbc4 80657bd4 8339bdb8 00028000 00006000 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.703359] bdc0: bbb2d690 8339be00 8339bdfc 8339bdd8 802bee10 802bdb60 9b118d80 8084da00
Mar 29 10:48:02 rasp3 kernel: [36470.706869] bde0: 00000000 00000000 00010000 01480000 8339be44 8339be00 802bf5d0 802bed08
Mar 29 10:48:02 rasp3 kernel: [36470.710384] be00: 0000a000 00001000 00000004 bc243f00 00006000 00000000 00000000 00006000
Mar 29 10:48:02 rasp3 kernel: [36470.713898] be20: 00000000 00000000 00000004 8084da00 01480000 00000000 8339be5c 8339be48
Mar 29 10:48:02 rasp3 kernel: [36470.717410] be40: 802bf618 802bf57c 00000004 802bdb54 8339be7c 8339be60 802bdb4c 802bf5f4
Mar 29 10:48:02 rasp3 kernel: [36470.720913] be60: 00000004 01480000 9b118d80 8339bed0 8339becc 8339be80 802be3f4 802bdb0c
Mar 29 10:48:02 rasp3 kernel: [36470.724429] be80: 00000000 80000007 00000000 802bdb00 00000000 b9fd4300 01490000 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.727942] bea0: 8078a67c 00000000 00000000 8339bf38 b9fd4300 bc243f00 bc243f00 0abe3a61
Mar 29 10:48:02 rasp3 kernel: [36470.731452] bec0: 8339bf14 8339bed0 802be618 802be314 00010000 0abe3a61 00000004 bc243f00
Mar 29 10:48:02 rasp3 kernel: [36470.734961] bee0: 01480000 00000000 8339bf40 00000000 00000000 00000000 b9fd4300 b9fd4300
Mar 29 10:48:02 rasp3 kernel: [36470.738466] bf00: 00000000 8339bf80 8339bf74 8339bf18 80289fd4 802be588 0abe3a61 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.741984] bf20: 00000020 00000000 bb3a3de0 b9fd4300 00000000 00000000 00000000 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.745500] bf40: 00000000 00000000 0000000b 7ef27910 0abe3a61 00000000 00000000 80108224
Mar 29 10:48:02 rasp3 kernel: [36470.749017] bf60: 8339a000 00000000 8339bfa4 8339bf78 8028b5d0 80289e24 fffff000 000001ff
Mar 29 10:48:02 rasp3 kernel: [36470.752527] bf80: 00000000 00000000 7ffffffe 00000000 00000000 000000ef 00000000 8339bfa8
Mar 29 10:48:02 rasp3 kernel: [36470.756035] bfa0: 80108080 8028b554 7ffffffe 00000000 0000000a 0000000b 7ef27910 0abe3a61
Mar 29 10:48:02 rasp3 kernel: [36470.759548] bfc0: 7ffffffe 00000000 00000000 000000ef 009074d8 00000000 0abe3a60 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.763061] bfe0: 76966340 7ef278ec 00018380 7696634c 60000010 0000000a 00000000 00000000
Mar 29 10:48:02 rasp3 kernel: [36470.766586] [<806ce3a4>] (tcp_push) from [<806d1950>] (tcp_sendmsg_locked+0x308/0xdfc)
Mar 29 10:48:02 rasp3 kernel: [36470.770095] [<806d1950>] (tcp_sendmsg_locked) from [<80657050>] (kernel_sendmsg_locked+0x60/0x78)
Mar 29 10:48:02 rasp3 kernel: [36470.773629] [<80657050>] (kernel_sendmsg_locked) from [<8065b568>] (sock_no_sendpage_locked+0xc8/0xd0)
Mar 29 10:48:02 rasp3 kernel: [36470.777175] [<8065b568>] (sock_no_sendpage_locked) from [<806d1584>] (tcp_sendpage_locked+0x70/0xd0)
Mar 29 10:48:02 rasp3 kernel: [36470.780730] [<806d1584>] (tcp_sendpage_locked) from [<806d1630>] (tcp_sendpage+0x4c/0x64)
Mar 29 10:48:02 rasp3 kernel: [36470.784281] [<806d1630>] (tcp_sendpage) from [<806fef58>] (inet_sendpage+0x5c/0x114)
Mar 29 10:48:02 rasp3 kernel: [36470.787821] [<806fef58>] (inet_sendpage) from [<80657bac>] (kernel_sendpage+0x34/0x50)
Mar 29 10:48:02 rasp3 kernel: [36470.791381] [<80657bac>] (kernel_sendpage) from [<80657c04>] (sock_sendpage+0x3c/0x44)
Mar 29 10:48:02 rasp3 kernel: [36470.794951] [<80657c04>] (sock_sendpage) from [<802bdbc4>] (pipe_to_sendpage+0x70/0x94)
Mar 29 10:48:02 rasp3 kernel: [36470.798526] [<802bdbc4>] (pipe_to_sendpage) from [<802bee10>] (__splice_from_pipe+0x114/0x1a8)
Mar 29 10:48:02 rasp3 kernel: [36470.802111] [<802bee10>] (__splice_from_pipe) from [<802bf5d0>] (splice_from_pipe+0x60/0x78)
Mar 29 10:48:02 rasp3 kernel: [36470.805692] [<802bf5d0>] (splice_from_pipe) from [<802bf618>] (generic_splice_sendpage+0x30/0x38)
Mar 29 10:48:02 rasp3 kernel: [36470.809288] [<802bf618>] (generic_splice_sendpage) from [<802bdb4c>] (direct_splice_actor+0x4c/0x54)
Mar 29 10:48:02 rasp3 kernel: [36470.812886] [<802bdb4c>] (direct_splice_actor) from [<802be3f4>] (splice_direct_to_actor+0xec/0x274)
Mar 29 10:48:02 rasp3 kernel: [36470.816485] [<802be3f4>] (splice_direct_to_actor) from [<802be618>] (do_splice_direct+0x9c/0xc4)
Mar 29 10:48:02 rasp3 kernel: [36470.820073] [<802be618>] (do_splice_direct) from [<80289fd4>] (do_sendfile+0x1bc/0x3e0)
Mar 29 10:48:02 rasp3 kernel: [36470.823658] [<80289fd4>] (do_sendfile) from [<8028b5d0>] (SyS_sendfile64+0x88/0xf8)
Mar 29 10:48:02 rasp3 kernel: [36470.827244] [<8028b5d0>] (SyS_sendfile64) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 29 10:48:02 rasp3 kernel: [36470.830836] Code: e5840514 e3110001 e59b1004 158404a4 (e595205c)
Mar 29 10:48:02 rasp3 kernel: [36470.832713] ---[ end trace 06173a3527f2531a ]---
pelwell
commented
6 years ago A patch for that bug was applied to our current branch this afternoon. The latest firmware build, available via rpi-update, includes the patch.
pelwell
commented
6 years ago Correction - the patch actually went in yesterday, and the firmware was also available yesterday. Your kernel is very recent, but I'm hoping it was from the release before the one with the fix.
zerwes
commented
6 years ago # cat /sys/firmware/devicetree/base/model
Raspberry Pi 3 Model B Rev 1.2
# cat /proc/cpuinfo
processor : 0
model name : ARMv7 Processor rev 4 (v7l)
BogoMIPS : 38.40
Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4
processor : 1
model name : ARMv7 Processor rev 4 (v7l)
BogoMIPS : 38.40
Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4
processor : 2
model name : ARMv7 Processor rev 4 (v7l)
BogoMIPS : 38.40
Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4
processor : 3
model name : ARMv7 Processor rev 4 (v7l)
BogoMIPS : 38.40
Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4
Hardware : BCM2835
Revision : a02082
Serial : 0000000062870c64
# lshw
rasp3
description: ARMv7 Processor rev 4 (v7l)
product: Raspberry Pi 3 Model B Rev 1.2
serial: 0000000062870c64
width: 32 bits
capabilities: smp
*-core
description: Motherboard
physical id: 0
*-cpu:0
description: CPU
product: cpu
physical id: 0
bus info: cpu@0
size: 600MHz
capacity: 1200MHz
capabilities: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 cpufreq
*-cpu:1
description: CPU
product: cpu
physical id: 1
bus info: cpu@1
size: 600MHz
capacity: 1200MHz
capabilities: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 cpufreq
*-cpu:2
description: CPU
product: cpu
physical id: 2
bus info: cpu@2
size: 600MHz
capacity: 1200MHz
capabilities: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 cpufreq
*-cpu:3
description: CPU
product: cpu
physical id: 3
bus info: cpu@3
size: 600MHz
capacity: 1200MHz
capabilities: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 cpufreq
*-memory
description: System memory
physical id: 4
size: 970MiB
*-usbhost
product: DWC OTG Controller
vendor: Linux 4.14.30-v7+ dwc_otg_hcd
physical id: 1
bus info: usb@1
logical name: usb1
version: 4.14
capabilities: usb-2.00
configuration: driver=hub slots=1 speed=480Mbit/s
*-usb
description: USB hub
product: SMC9514 Hub
vendor: Standard Microsystems Corp.
physical id: 1
bus info: usb@1:1
version: 2.00
capabilities: usb-2.00
configuration: driver=hub maxpower=2mA slots=5 speed=480Mbit/s
*-usb:0
description: Generic USB device
product: SMSC9512/9514 Fast Ethernet Adapter
vendor: Standard Microsystems Corp.
physical id: 1
bus info: usb@1:1.1
version: 2.00
capabilities: usb-2.00
configuration: driver=smsc95xx maxpower=2mA speed=480Mbit/s
*-usb:1
description: Mass storage device
product: Elements 25A2
vendor: Western Digital
physical id: 5
bus info: usb@1:1.5
logical name: scsi0
version: 10.14
serial: 575834314131374437485550
capabilities: usb-2.10 scsi emulated
configuration: driver=usb-storage maxpower=500mA speed=480Mbit/s
*-disk
description: SCSI Disk
product: Elements 25A2
vendor: WD
physical id: 0.0.0
bus info: scsi@0:0.0.0
logical name: /dev/sda
version: 1014
serial: WX41A17D7HUP
size: 931GiB (1TB)
capabilities: gpt-1.00 partitioned partitioned:gpt
configuration: ansiversion=6 guid=05f3eea3-1abb-4771-9676-da24e2b8f8be logicalsectorsize=512 sectorsize=512
*-volume
description: EXT3 volume
vendor: Linux
physical id: 1
bus info: scsi@0:0.0.0,1
logical name: /dev/sda1
logical name: /srv
version: 1.0
serial: 6cb824fb-4852-4462-8208-cd894b844e09
size: 931GiB
capacity: 931GiB
capabilities: journaled extended_attributes large_files recover ext3 ext2 initialized
configuration: created=2017-02-23 23:52:08 filesystem=ext3 label=data lastmountpoint=/srv modified=2018-03-29 13:52:22 mount.fstype=ext4 mount.options=rw,relatime,data=ordered mounted=2018-03-29 13:52:22 state=mounted
*-network:0
description: Ethernet interface
physical id: 2
logical name: eth0
serial: b8:27:eb:87:0c:64
size: 100Mbit/s
capacity: 100Mbit/s
capabilities: ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=smsc95xx driverversion=22-Aug-2005 duplex=full firmware=smsc95xx USB 2.0 Ethernet ip=192.168.100.20 link=yes multicast=yes port=MII speed=100Mbit/s
*-network:1
description: Wireless interface
physical id: 3
logical name: wlan0
serial: b8:27:eb:d2:59:31
capabilities: ethernet physical wireless
configuration: broadcast=yes driver=brcmfmac driverversion=7.45.98.38 firmware=01-e58d219f multicast=yes wireless=IEEE 802.11
# lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description: Raspbian GNU/Linux 9.4 (stretch)
Release: 9.4
Codename: stretch
# dpkg -l raspberrypi-kernel
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=====================================-=======================-=======================-===============================================================================
ii raspberrypi-kernel 1.20180328-1 armhf Raspberry Pi bootloadercrashes appear after:
Start-Date: 2018-03-29 00:36:21
Commandline: apt-get -y dist-upgrade
Upgrade: libraspberrypi-bin:armhf (1.20180313-1, 1.20180328-1), libraspberrypi-dev:armhf (1.20180313-1, 1.20180328-1), raspberrypi-kernel:armhf (1.20180313-1, 1.20180328-1), raspberrypi-bootloader:armhf (1.20180313-1, 1.20180328-1), libraspberrypi0:armhf (1.20180313-1, 1.20180328-1), raspberrypi-sys-mods:armhf (20180315, 20180328+1)
End-Date: 2018-03-29 00:38:42downgrade not possible using apt-get:
# apt-get install libraspberrypi-bin=1.20180313-1 libraspberrypi-dev=1.20180313-1 raspberrypi-kernel=1.20180313-1 raspberrypi-bootloader=1.20180313-1 libraspberrypi0=1.20180313-1 raspberrypi-sys-mods=20180315
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Version '1.20180313-1' for 'libraspberrypi-bin' was not found
E: Version '1.20180313-1' for 'libraspberrypi-dev' was not found
E: Version '1.20180313-1' for 'raspberrypi-kernel' was not found
E: Version '1.20180313-1' for 'raspberrypi-bootloader' was not found
E: Version '1.20180313-1' for 'libraspberrypi0' was not found
E: Version '20180315' for 'raspberrypi-sys-mods' was not foundusing archived versions:
root@rasp3:/var/cache/apt/archives# dpkg -i libraspberrypi-bin*1.20180313-1* libraspberrypi-dev*1.20180313-1* raspberrypi-kernel*1.20180313-1* raspberrypi-bootloader*1.20180313-1* libraspberrypi0*1.20180313-1* raspberrypi-sys-mods*20180315*# dpkg -l libraspberrypi-bin libraspberrypi-dev raspberrypi-kernel raspberrypi-bootloader libraspberrypi0 raspberrypi-sys-mods Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=====================================-=======================-=======================-===============================================================================
ii libraspberrypi-bin 1.20180313-1 armhf Miscellaneous Raspberry Pi utilities
ii libraspberrypi-dev 1.20180313-1 armhf EGL/GLES/OpenVG/etc. libraries for the Raspberry Pi's VideoCore IV (headers)
ii libraspberrypi0 1.20180313-1 armhf EGL/GLES/OpenVG/etc. libraries for the Raspberry Pi's VideoCore IV
ii raspberrypi-bootloader 1.20180313-1 armhf Raspberry Pi bootloader
ii raspberrypi-kernel 1.20180313-1 armhf Raspberry Pi bootloader
ii raspberrypi-sys-mods 20180315 armhf System tweaks for the Raspberry PiUpgrade by running sudo rpi-update.
teddythomas
commented
6 years ago It looks like the patch is going to be merged to the stable 4.14 tree as part of the 4.14.32 release (https://www.spinics.net/lists/stable/msg230368.html) š
comdata
commented
6 years ago Ok, so I will wait for the 4.14.32 then. Thanks.
horgh
commented
6 years ago I think I'm experiencing this too. I've been posting about it in this thread on the forum: https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=208821
With the latest firmware, I'm still seeing crashes:
pi@rpi5:~ $ cat /boot/.firmware_revision
2148253d340d7c646583cb8f0be6403511522cc4Mar 30 22:49:01 rpi5 kernel: [33987.220493] Alignment trap: not handling instruction e1903f9f at [<8079ef70>]
Mar 30 22:49:01 rpi5 kernel: [33987.220510] Unhandled fault: alignment exception (0x001) at 0xba4429ed
Mar 30 22:49:01 rpi5 kernel: [33987.222504] pgd = b9f7c000
Mar 30 22:49:01 rpi5 kernel: [33987.224400] [ba4429ed] *pgd=3a41141e(bad)
Mar 30 22:49:01 rpi5 kernel: [33987.226328] Internal error: : 1 [#1] SMP ARM
Mar 30 22:49:01 rpi5 kernel: [33987.228248] Modules linked in: ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter sg brcmfmac brcmutil cfg80211 rfkill snd_bcm2835(C) snd_pcm snd_timer snd uio_pdrv_genirq uio fixed ip_tables x_tables ipv6
Mar 30 22:49:01 rpi5 kernel: [33987.237083] CPU: 1 PID: 7877 Comm: sshd Tainted: G C 4.14.31-v7+ #1104
Mar 30 22:49:01 rpi5 kernel: [33987.241838] Hardware name: BCM2835
Mar 30 22:49:01 rpi5 kernel: [33987.244212] task: b72e8f00 task.stack: b9c68000
Mar 30 22:49:01 rpi5 kernel: [33987.246637] PC is at _raw_read_lock+0x1c/0x38
Mar 30 22:49:01 rpi5 kernel: [33987.249041] LR is at ext4_es_lookup_extent+0x30/0x244
Mar 30 22:49:01 rpi5 kernel: [33987.251491] pc : [<8079ef74>] lr : [<803265ac>] psr: 00000013
Mar 30 22:49:01 rpi5 kernel: [33987.254016] sp : b9c69cd0 ip : b9c69ce0 fp : b9c69cdc
Mar 30 22:49:01 rpi5 kernel: [33987.256529] r10: 00000000 r9 : 00001000 r8 : ba4429ed
Mar 30 22:49:01 rpi5 kernel: [33987.259030] r7 : 00000000 r6 : 427e0000 r5 : b9c69d28 r4 : ba442861
Mar 30 22:49:01 rpi5 kernel: [33987.261558] r3 : 00000000 r2 : b9c69d28 r1 : 427e0000 r0 : ba4429ed
Mar 30 22:49:01 rpi5 kernel: [33987.264064] Flags: nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Mar 30 22:49:01 rpi5 kernel: [33987.266663] Control: 10c5383d Table: 39f7c06a DAC: 00000055
Mar 30 22:49:01 rpi5 kernel: [33987.269221] Process sshd (pid: 7877, stack limit = 0xb9c68210)
Mar 30 22:49:01 rpi5 kernel: [33987.271769] Stack: (0xb9c69cd0 to 0xb9c6a000)
Mar 30 22:49:01 rpi5 kernel: [33987.274317] 9cc0: b9c69d04 b9c69ce0 803265ac 8079ef64
Mar 30 22:49:01 rpi5 kernel: [33987.279447] 9ce0: 00000000 b9c69dc0 ba442861 00000000 00000000 ba442860 b9c69d74 b9c69d08
Mar 30 22:49:01 rpi5 kernel: [33987.284516] 9d00: 80335f60 80326588 40000013 ffffffff b9c69d5c ba442860 b9c69e44 b9c69d28
Mar 30 22:49:01 rpi5 kernel: [33987.289597] 9d20: 8079f680 801eda18 80ffffec ba442861 00001000 00000000 10000022 b2000001
Mar 30 22:49:01 rpi5 kernel: [33987.294868] 9d40: 427e0000 00000ba4 ba442860 00000000 00001000 427e0000 00000ba4 ba442860
Mar 30 22:49:01 rpi5 kernel: [33987.300470] 9d60: 00001000 00000000 b9c69e44 b9c69d78 80355cf4 80335f14 80355868 80d162ac
Mar 30 22:49:01 rpi5 kernel: [33987.306363] 9d80: 00000001 00000000 00000000 00000000 78daaef1 ba442860 7f000000 00000000
Mar 30 22:49:01 rpi5 kernel: [33987.312502] 9da0: 00000000 00000001 00001000 ba442861 00000001 00000ba4 427e1000 00000ba4
Mar 30 22:49:01 rpi5 kernel: [33987.318979] 9dc0: 00000000 00000000 427e0000 00001000 00000000 00000000 00000000 b640e820
Mar 30 22:49:01 rpi5 kernel: [33987.325631] 9de0: b9c69e04 b9c69df0 8078b708 8078b644 b9c69e1c ba442864 b9c69e2c b9c69e08
Mar 30 22:49:01 rpi5 kernel: [33987.332468] 9e00: 8021d0d8 8078b6f8 00000000 00000000 00000000 00000000 b640e81c 00000000
Mar 30 22:49:01 rpi5 kernel: [33987.339453] 9e20: ba442860 ba442861 b9d94f00 b9c69ef0 b9c69f08 ba442860 b9c69e64 b9c69e48
Mar 30 22:49:01 rpi5 kernel: [33987.346512] 9e40: 80334270 80355914 80334204 00000000 ba442860 b640e81c b9c69ecc b9c69e68
Mar 30 22:49:01 rpi5 kernel: [33987.353571] 9e60: 8022036c 80334210 b97ad6e8 b9c69e78 8029a528 8029a178 00080001 00000000
Mar 30 22:49:01 rpi5 kernel: [33987.360629] 9e80: 00000001 ffffffff b9d94f68 00000fff b640e730 b9c68000 00000000 b9c69ea8
Mar 30 22:49:01 rpi5 kernel: [33987.367679] 9ea0: 8029afa0 b9d94f00 b9c69f80 00000000 00000000 00000200 00000000 00000000
Mar 30 22:49:01 rpi5 kernel: [33987.374733] 9ec0: b9c69edc b9c69ed0 80327bc4 8021fe9c b9c69f4c b9c69ee0 8028a6cc 80327b8c
Mar 30 22:49:01 rpi5 kernel: [33987.381303] 9ee0: 00000200 b9c69ee8 7ea75670 00000200 00000000 00000000 00000200 b9c69ee8
Mar 30 22:49:01 rpi5 kernel: [33987.384329] 9f00: 00000001 8079a468 b9d94f00 00000000 00000000 00000000 00000000 00000000
Mar 30 22:49:01 rpi5 kernel: [33987.387358] 9f20: 00000000 00000000 00000200 b9d94f00 7ea75670 b9c69f80 80108224 b9c68000
Mar 30 22:49:01 rpi5 kernel: [33987.390380] 9f40: b9c69f7c b9c69f50 8028a7b8 8028a5f8 00000000 802aadbc b9c69f7c b9d94f00
Mar 30 22:49:01 rpi5 kernel: [33987.393401] 9f60: b9d94f00 00000200 7ea75670 80108224 b9c69fa4 b9c69f80 8028ad68 8028a728
Mar 30 22:49:01 rpi5 kernel: [33987.396429] 9f80: 00000000 00000000 00012e57 00000008 76fd3960 00000003 00000000 b9c69fa8
Mar 30 22:49:01 rpi5 kernel: [33987.399457] 9fa0: 80108080 8028ad20 00012e57 00000008 00000003 7ea75670 00000200 00000000
Mar 30 22:49:01 rpi5 kernel: [33987.402486] 9fc0: 00012e57 00000008 76fd3960 00000003 00000003 7ea75670 76fd3b48 7ea75624
Mar 30 22:49:01 rpi5 kernel: [33987.405515] 9fe0: 76fd2cf0 7ea755cc 76fa92c0 76fbc4ec 00000010 00000003 00000000 00000000
Mar 30 22:49:01 rpi5 kernel: [33987.408543] [<8079ef74>] (_raw_read_lock) from [<803265ac>] (ext4_es_lookup_extent+0x30/0x244)
Mar 30 22:49:01 rpi5 kernel: [33987.411583] [<803265ac>] (ext4_es_lookup_extent) from [<80335f60>] (ext4_map_blocks+0x58/0x610)
Mar 30 22:49:01 rpi5 kernel: [33987.414636] [<80335f60>] (ext4_map_blocks) from [<80355cf4>] (ext4_mpage_readpages+0x3ec/0x8b4)
Mar 30 22:49:01 rpi5 kernel: [33987.417692] [<80355cf4>] (ext4_mpage_readpages) from [<80334270>] (ext4_readpage+0x6c/0xe0)
Mar 30 22:49:01 rpi5 kernel: [33987.420753] [<80334270>] (ext4_readpage) from [<8022036c>] (generic_file_read_iter+0x4dc/0x9d0)
Mar 30 22:49:01 rpi5 kernel: [33987.423815] [<8022036c>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 22:49:01 rpi5 kernel: [33987.426892] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 22:49:01 rpi5 kernel: [33987.429967] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 22:49:01 rpi5 kernel: [33987.431544] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 22:49:01 rpi5 kernel: [33987.433105] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 22:49:01 rpi5 kernel: [33987.436081] Code: e52de004 e8bd4000 f590f000 e1903f9f (e2933001)
Mar 30 22:49:01 rpi5 kernel: [33987.437639] ---[ end trace 782b237550c7e2e4 ]---
Mar 30 22:53:22 rpi5 kernel: [34248.668444] INFO: rcu_sched self-detected stall on CPU
Mar 30 22:53:22 rpi5 kernel: [34248.669925] 2-...: (2099 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=1050
Mar 30 22:53:22 rpi5 kernel: [34248.672768] (t=2100 jiffies g=139900 c=139899 q=298)
Mar 30 22:53:22 rpi5 kernel: [34248.674183] NMI backtrace for cpu 2
Mar 30 22:53:22 rpi5 kernel: [34248.675550] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 22:53:22 rpi5 kernel: [34248.678225] Hardware name: BCM2835
Mar 30 22:53:22 rpi5 kernel: [34248.679529] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 22:53:22 rpi5 kernel: [34248.682099] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 22:53:22 rpi5 kernel: [34248.683427] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 22:53:22 rpi5 kernel: [34248.685921] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 22:53:22 rpi5 kernel: [34248.688438] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 22:53:22 rpi5 kernel: [34248.691044] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 22:53:22 rpi5 kernel: [34248.693689] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 22:53:22 rpi5 kernel: [34248.696406] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 22:53:22 rpi5 kernel: [34248.699166] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 22:53:22 rpi5 kernel: [34248.701989] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 22:53:22 rpi5 kernel: [34248.704893] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 22:53:22 rpi5 kernel: [34248.707910] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 22:53:22 rpi5 kernel: [34248.710935] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 22:53:22 rpi5 kernel: [34248.714032] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 22:53:22 rpi5 kernel: [34248.717133] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 22:53:22 rpi5 kernel: [34248.720217] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 22:53:22 rpi5 kernel: [34248.723307] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 22:53:22 rpi5 kernel: [34248.726410] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 22:53:22 rpi5 kernel: [34248.729502] Exception stack(0xb9eddd80 to 0xb9edddc8)
Mar 30 22:53:22 rpi5 kernel: [34248.731036] dd80: b97c9a60 00000000 00000000 b9edddf4 ba443c81 00000040 ba443c81 ba443c94
Mar 30 22:53:22 rpi5 kernel: [34248.734050] dda0: 80c045e8 00000000 80c8826c b9edddec bb3c3400 b9edddd0 ba443c80 8078b6a8
Mar 30 22:53:22 rpi5 kernel: [34248.737052] ddc0: a0000013 ffffffff
Mar 30 22:53:22 rpi5 kernel: [34248.738521] [<8079f6fc>] (__irq_svc) from [<8078b6a8>] (__radix_tree_lookup+0x70/0xb4)
Mar 30 22:53:22 rpi5 kernel: [34248.741435] [<8078b6a8>] (__radix_tree_lookup) from [<8078b708>] (radix_tree_lookup_slot+0x1c/0x2c)
Mar 30 22:53:22 rpi5 kernel: [34248.744365] [<8078b708>] (radix_tree_lookup_slot) from [<8021d0d8>] (find_get_entry+0x30/0x110)
Mar 30 22:53:22 rpi5 kernel: [34248.747291] [<8021d0d8>] (find_get_entry) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 22:53:22 rpi5 kernel: [34248.750210] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 22:53:22 rpi5 kernel: [34248.753131] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 22:53:22 rpi5 kernel: [34248.756061] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 22:53:22 rpi5 kernel: [34248.758980] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 22:53:22 rpi5 kernel: [34248.760499] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 22:53:22 rpi5 kernel: [34248.762006] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 22:54:25 rpi5 kernel: [34311.698876] INFO: rcu_sched self-detected stall on CPU
Mar 30 22:54:25 rpi5 kernel: [34311.700343] 2-...: (8394 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=4196
Mar 30 22:54:25 rpi5 kernel: [34311.703255] (t=8403 jiffies g=139900 c=139899 q=1045)
Mar 30 22:54:25 rpi5 kernel: [34311.704708] NMI backtrace for cpu 2
Mar 30 22:54:25 rpi5 kernel: [34311.706113] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 22:54:25 rpi5 kernel: [34311.708866] Hardware name: BCM2835
Mar 30 22:54:25 rpi5 kernel: [34311.710211] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 22:54:25 rpi5 kernel: [34311.712862] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 22:54:25 rpi5 kernel: [34311.714228] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 22:54:25 rpi5 kernel: [34311.716809] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 22:54:25 rpi5 kernel: [34311.719409] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 22:54:25 rpi5 kernel: [34311.722030] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 22:54:25 rpi5 kernel: [34311.724709] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 22:54:25 rpi5 kernel: [34311.727437] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 22:54:25 rpi5 kernel: [34311.730197] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 22:54:25 rpi5 kernel: [34311.733025] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 22:54:25 rpi5 kernel: [34311.735938] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 22:54:25 rpi5 kernel: [34311.738954] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 22:54:25 rpi5 kernel: [34311.741981] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 22:54:25 rpi5 kernel: [34311.745081] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 22:54:25 rpi5 kernel: [34311.748178] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 22:54:25 rpi5 kernel: [34311.751271] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 22:54:25 rpi5 kernel: [34311.754365] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 22:54:25 rpi5 kernel: [34311.757469] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 22:54:25 rpi5 kernel: [34311.760558] Exception stack(0xb9edddb8 to 0xb9edde00)
Mar 30 22:54:25 rpi5 kernel: [34311.762091] dda0: bb3c3400 ba443c94
Mar 30 22:54:25 rpi5 kernel: [34311.765027] ddc0: bb3c3410 bb3c3400 00000001 00000000 00000000 b97c9a60 80c045e8 00000000
Mar 30 22:54:25 rpi5 kernel: [34311.767919] dde0: 80c8826c b9edde2c 00000000 b9edde08 ba443c80 8021d12c 60000013 ffffffff
Mar 30 22:54:25 rpi5 kernel: [34311.770814] [<8079f6fc>] (__irq_svc) from [<8021d12c>] (find_get_entry+0x84/0x110)
Mar 30 22:54:25 rpi5 kernel: [34311.773700] [<8021d12c>] (find_get_entry) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 22:54:25 rpi5 kernel: [34311.776591] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 22:54:25 rpi5 kernel: [34311.779496] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 22:54:25 rpi5 kernel: [34311.782389] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 22:54:25 rpi5 kernel: [34311.785280] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 22:54:25 rpi5 kernel: [34311.786762] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 22:54:25 rpi5 kernel: [34311.788222] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 22:55:28 rpi5 kernel: [34374.729308] INFO: rcu_sched self-detected stall on CPU
Mar 30 22:55:28 rpi5 kernel: [34374.730771] 2-...: (14689 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=7342
Mar 30 22:55:28 rpi5 kernel: [34374.733680] (t=14706 jiffies g=139900 c=139899 q=1853)
Mar 30 22:55:28 rpi5 kernel: [34374.735133] NMI backtrace for cpu 2
Mar 30 22:55:28 rpi5 kernel: [34374.736536] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 22:55:28 rpi5 kernel: [34374.739282] Hardware name: BCM2835
Mar 30 22:55:28 rpi5 kernel: [34374.740624] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 22:55:28 rpi5 kernel: [34374.743268] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 22:55:28 rpi5 kernel: [34374.744632] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 22:55:28 rpi5 kernel: [34374.747208] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 22:55:28 rpi5 kernel: [34374.749804] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 22:55:28 rpi5 kernel: [34374.752413] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 22:55:28 rpi5 kernel: [34374.755091] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 22:55:28 rpi5 kernel: [34374.757818] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 22:55:28 rpi5 kernel: [34374.760577] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 22:55:28 rpi5 kernel: [34374.763406] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 22:55:28 rpi5 kernel: [34374.766317] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 22:55:28 rpi5 kernel: [34374.769333] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 22:55:28 rpi5 kernel: [34374.772360] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 22:55:28 rpi5 kernel: [34374.775456] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 22:55:28 rpi5 kernel: [34374.778548] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 22:55:28 rpi5 kernel: [34374.781641] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 22:55:28 rpi5 kernel: [34374.784730] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 22:55:28 rpi5 kernel: [34374.787833] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 22:55:28 rpi5 kernel: [34374.790924] Exception stack(0xb9edddb8 to 0xb9edde00)
Mar 30 22:55:28 rpi5 kernel: [34374.792455] dda0: bb3c3400 ba443c94
Mar 30 22:55:28 rpi5 kernel: [34374.795386] ddc0: bb3c3410 bb3c3400 00000001 00000000 00000000 b97c9a60 80c045e8 00000000
Mar 30 22:55:28 rpi5 kernel: [34374.798275] dde0: 80c8826c b9edde2c bb3c3400 b9edde08 ba443c80 8021d108 60000013 ffffffff
Mar 30 22:55:28 rpi5 kernel: [34374.801165] [<8079f6fc>] (__irq_svc) from [<8021d108>] (find_get_entry+0x60/0x110)
Mar 30 22:55:28 rpi5 kernel: [34374.804046] [<8021d108>] (find_get_entry) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 22:55:28 rpi5 kernel: [34374.806937] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 22:55:28 rpi5 kernel: [34374.809834] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 22:55:28 rpi5 kernel: [34374.812725] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 22:55:28 rpi5 kernel: [34374.815613] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 22:55:28 rpi5 kernel: [34374.817094] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 22:55:28 rpi5 kernel: [34374.818552] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 22:56:31 rpi5 kernel: [34437.759741] INFO: rcu_sched self-detected stall on CPU
Mar 30 22:56:31 rpi5 kernel: [34437.761202] 2-...: (20984 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=10488
Mar 30 22:56:31 rpi5 kernel: [34437.764115] (t=21009 jiffies g=139900 c=139899 q=2529)
Mar 30 22:56:31 rpi5 kernel: [34437.765566] NMI backtrace for cpu 2
Mar 30 22:56:31 rpi5 kernel: [34437.766970] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 22:56:31 rpi5 kernel: [34437.769722] Hardware name: BCM2835
Mar 30 22:56:31 rpi5 kernel: [34437.771066] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 22:56:31 rpi5 kernel: [34437.773710] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 22:56:31 rpi5 kernel: [34437.775075] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 22:56:31 rpi5 kernel: [34437.777659] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 22:56:31 rpi5 kernel: [34437.780259] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 22:56:31 rpi5 kernel: [34437.782873] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 22:56:31 rpi5 kernel: [34437.785554] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 22:56:31 rpi5 kernel: [34437.788282] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 22:56:31 rpi5 kernel: [34437.791039] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 22:56:31 rpi5 kernel: [34437.793875] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 22:56:31 rpi5 kernel: [34437.796787] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 22:56:31 rpi5 kernel: [34437.799802] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 22:56:31 rpi5 kernel: [34437.802829] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 22:56:31 rpi5 kernel: [34437.805925] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 22:56:31 rpi5 kernel: [34437.809017] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 22:56:31 rpi5 kernel: [34437.812115] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 22:56:31 rpi5 kernel: [34437.815208] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 22:56:31 rpi5 kernel: [34437.818313] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 22:56:31 rpi5 kernel: [34437.821402] Exception stack(0xb9eddda8 to 0xb9edddf0)
Mar 30 22:56:31 rpi5 kernel: [34437.822932] dda0: b97c9a60 00000000 bb3c3410 bb3c3400 00000001 00000000
Mar 30 22:56:31 rpi5 kernel: [34437.825924] ddc0: 00000000 b97c9a60 80c045e8 00000000 80c8826c b9edde2c b9edde08 b9edddf8
Mar 30 22:56:31 rpi5 kernel: [34437.828905] dde0: 8021d0d8 8078b6f4 60000013 ffffffff
Mar 30 22:56:31 rpi5 kernel: [34437.830391] [<8079f6fc>] (__irq_svc) from [<8078b6f4>] (radix_tree_lookup_slot+0x8/0x2c)
Mar 30 22:56:31 rpi5 kernel: [34437.833315] [<8078b6f4>] (radix_tree_lookup_slot) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 22:56:31 rpi5 kernel: [34437.836249] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 22:56:31 rpi5 kernel: [34437.839183] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 22:56:31 rpi5 kernel: [34437.842108] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 22:56:31 rpi5 kernel: [34437.845032] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 22:56:31 rpi5 kernel: [34437.846529] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 22:56:31 rpi5 kernel: [34437.848008] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 22:57:34 rpi5 kernel: [34500.790174] INFO: rcu_sched self-detected stall on CPU
Mar 30 22:57:34 rpi5 kernel: [34500.791632] 2-...: (27279 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=13633
Mar 30 22:57:34 rpi5 kernel: [34500.794470] (t=27312 jiffies g=139900 c=139899 q=3411)
Mar 30 22:57:34 rpi5 kernel: [34500.795891] NMI backtrace for cpu 2
Mar 30 22:57:34 rpi5 kernel: [34500.797265] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 22:57:34 rpi5 kernel: [34500.800008] Hardware name: BCM2835
Mar 30 22:57:34 rpi5 kernel: [34500.801352] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 22:57:34 rpi5 kernel: [34500.803994] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 22:57:34 rpi5 kernel: [34500.805359] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 22:57:34 rpi5 kernel: [34500.807943] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 22:57:34 rpi5 kernel: [34500.810542] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 22:57:34 rpi5 kernel: [34500.813156] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 22:57:34 rpi5 kernel: [34500.815834] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 22:57:34 rpi5 kernel: [34500.818561] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 22:57:34 rpi5 kernel: [34500.821318] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 22:57:34 rpi5 kernel: [34500.824157] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 22:57:34 rpi5 kernel: [34500.827070] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 22:57:34 rpi5 kernel: [34500.830084] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 22:57:34 rpi5 kernel: [34500.833113] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 22:57:34 rpi5 kernel: [34500.836206] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 22:57:34 rpi5 kernel: [34500.839298] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 22:57:34 rpi5 kernel: [34500.842394] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 22:57:34 rpi5 kernel: [34500.845485] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 22:57:34 rpi5 kernel: [34500.848587] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 22:57:34 rpi5 kernel: [34500.851673] Exception stack(0xb9eddda8 to 0xb9edddf0)
Mar 30 22:57:34 rpi5 kernel: [34500.853200] dda0: b97c9a60 00000000 bb3c3410 bb3c3400 00000001 00000000
Mar 30 22:57:34 rpi5 kernel: [34500.856195] ddc0: 00000000 b97c9a60 80c045e8 00000000 80c8826c b9edde2c b9edde08 b9edddf8
Mar 30 22:57:34 rpi5 kernel: [34500.859174] dde0: 8021d0d8 8078b6f4 60000013 ffffffff
Mar 30 22:57:34 rpi5 kernel: [34500.860659] [<8079f6fc>] (__irq_svc) from [<8078b6f4>] (radix_tree_lookup_slot+0x8/0x2c)
Mar 30 22:57:34 rpi5 kernel: [34500.863582] [<8078b6f4>] (radix_tree_lookup_slot) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 22:57:34 rpi5 kernel: [34500.866511] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 22:57:34 rpi5 kernel: [34500.869438] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 22:57:34 rpi5 kernel: [34500.872360] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 22:57:34 rpi5 kernel: [34500.875278] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 22:57:34 rpi5 kernel: [34500.876775] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 22:57:34 rpi5 kernel: [34500.878250] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 22:58:37 rpi5 kernel: [34563.820608] INFO: rcu_sched self-detected stall on CPU
Mar 30 22:58:37 rpi5 kernel: [34563.822063] 2-...: (33574 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=16779
Mar 30 22:58:37 rpi5 kernel: [34563.824892] (t=33615 jiffies g=139900 c=139899 q=4078)
Mar 30 22:58:37 rpi5 kernel: [34563.826316] NMI backtrace for cpu 2
Mar 30 22:58:37 rpi5 kernel: [34563.827690] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 22:58:37 rpi5 kernel: [34563.830433] Hardware name: BCM2835
Mar 30 22:58:37 rpi5 kernel: [34563.831775] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 22:58:37 rpi5 kernel: [34563.834417] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 22:58:37 rpi5 kernel: [34563.835784] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 22:58:37 rpi5 kernel: [34563.838370] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 22:58:37 rpi5 kernel: [34563.840969] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 22:58:37 rpi5 kernel: [34563.843582] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 22:58:37 rpi5 kernel: [34563.846263] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 22:58:37 rpi5 kernel: [34563.848987] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 22:58:37 rpi5 kernel: [34563.851750] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 22:58:37 rpi5 kernel: [34563.854586] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 22:58:37 rpi5 kernel: [34563.857500] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 22:58:37 rpi5 kernel: [34563.860515] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 22:58:37 rpi5 kernel: [34563.863540] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 22:58:37 rpi5 kernel: [34563.866629] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 22:58:37 rpi5 kernel: [34563.869726] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 22:58:37 rpi5 kernel: [34563.872820] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 22:58:37 rpi5 kernel: [34563.875909] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 22:58:37 rpi5 kernel: [34563.879013] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 22:58:37 rpi5 kernel: [34563.882097] Exception stack(0xb9edddb8 to 0xb9edde00)
Mar 30 22:58:37 rpi5 kernel: [34563.883625] dda0: bb3c3400 ba443c94
Mar 30 22:58:37 rpi5 kernel: [34563.886559] ddc0: 00000000 b9edddf4 00000001 00000000 00000000 b97c9a60 80c045e8 00000000
Mar 30 22:58:37 rpi5 kernel: [34563.889451] dde0: 80c8826c b9edde2c bb3c3400 b9edde08 ba443c80 8021d0ec a0000013 ffffffff
Mar 30 22:58:37 rpi5 kernel: [34563.892343] [<8079f6fc>] (__irq_svc) from [<8021d0ec>] (find_get_entry+0x44/0x110)
Mar 30 22:58:37 rpi5 kernel: [34563.895225] [<8021d0ec>] (find_get_entry) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 22:58:37 rpi5 kernel: [34563.898117] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 22:58:37 rpi5 kernel: [34563.901010] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 22:58:37 rpi5 kernel: [34563.903912] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 22:58:37 rpi5 kernel: [34563.906804] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 22:58:37 rpi5 kernel: [34563.908285] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 22:58:37 rpi5 kernel: [34563.909749] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 22:59:40 rpi5 kernel: [34626.851038] INFO: rcu_sched self-detected stall on CPU
Mar 30 22:59:40 rpi5 kernel: [34626.852499] 2-...: (39869 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=19925
Mar 30 22:59:40 rpi5 kernel: [34626.855414] (t=39918 jiffies g=139900 c=139899 q=4714)
Mar 30 22:59:40 rpi5 kernel: [34626.856868] NMI backtrace for cpu 2
Mar 30 22:59:40 rpi5 kernel: [34626.858271] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 22:59:40 rpi5 kernel: [34626.861021] Hardware name: BCM2835
Mar 30 22:59:40 rpi5 kernel: [34626.862360] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 22:59:40 rpi5 kernel: [34626.865004] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 22:59:40 rpi5 kernel: [34626.866374] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 22:59:40 rpi5 kernel: [34626.868956] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 22:59:40 rpi5 kernel: [34626.871554] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 22:59:40 rpi5 kernel: [34626.874165] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 22:59:40 rpi5 kernel: [34626.876841] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 22:59:40 rpi5 kernel: [34626.879558] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 22:59:40 rpi5 kernel: [34626.882319] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 22:59:40 rpi5 kernel: [34626.885154] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 22:59:40 rpi5 kernel: [34626.888065] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 22:59:40 rpi5 kernel: [34626.891080] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 22:59:40 rpi5 kernel: [34626.894108] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 22:59:40 rpi5 kernel: [34626.897197] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 22:59:40 rpi5 kernel: [34626.900299] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 22:59:40 rpi5 kernel: [34626.903396] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 22:59:40 rpi5 kernel: [34626.906490] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 22:59:40 rpi5 kernel: [34626.909597] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 22:59:40 rpi5 kernel: [34626.912680] Exception stack(0xb9eddd80 to 0xb9edddc8)
Mar 30 22:59:40 rpi5 kernel: [34626.914209] dd80: b97c9a60 00000000 00000000 b9edddf4 ba443c81 00000040 ba443c81 b97c9a60
Mar 30 22:59:40 rpi5 kernel: [34626.917229] dda0: 80c045e8 00000000 80c8826c b9edddec 0000003f b9edddd0 8078b708 8078b688
Mar 30 22:59:40 rpi5 kernel: [34626.920241] ddc0: a0000013 ffffffff
Mar 30 22:59:40 rpi5 kernel: [34626.921710] [<8079f6fc>] (__irq_svc) from [<8078b688>] (__radix_tree_lookup+0x50/0xb4)
Mar 30 22:59:40 rpi5 kernel: [34626.924630] [<8078b688>] (__radix_tree_lookup) from [<8078b708>] (radix_tree_lookup_slot+0x1c/0x2c)
Mar 30 22:59:40 rpi5 kernel: [34626.927560] [<8078b708>] (radix_tree_lookup_slot) from [<8021d0d8>] (find_get_entry+0x30/0x110)
Mar 30 22:59:40 rpi5 kernel: [34626.930476] [<8021d0d8>] (find_get_entry) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 22:59:40 rpi5 kernel: [34626.933396] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 22:59:40 rpi5 kernel: [34626.936328] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 22:59:40 rpi5 kernel: [34626.939259] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 22:59:40 rpi5 kernel: [34626.942181] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 22:59:40 rpi5 kernel: [34626.943701] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 22:59:40 rpi5 kernel: [34626.945204] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 23:00:43 rpi5 kernel: [34689.881474] INFO: rcu_sched self-detected stall on CPU
Mar 30 23:00:43 rpi5 kernel: [34689.882943] 2-...: (46164 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=23071
Mar 30 23:00:43 rpi5 kernel: [34689.885852] (t=46221 jiffies g=139900 c=139899 q=5543)
Mar 30 23:00:43 rpi5 kernel: [34689.887308] NMI backtrace for cpu 2
Mar 30 23:00:43 rpi5 kernel: [34689.888713] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 23:00:43 rpi5 kernel: [34689.891461] Hardware name: BCM2835
Mar 30 23:00:43 rpi5 kernel: [34689.892800] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 23:00:43 rpi5 kernel: [34689.895445] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 23:00:43 rpi5 kernel: [34689.896812] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 23:00:43 rpi5 kernel: [34689.899394] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 23:00:43 rpi5 kernel: [34689.901993] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 23:00:43 rpi5 kernel: [34689.904607] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 23:00:43 rpi5 kernel: [34689.907284] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 23:00:43 rpi5 kernel: [34689.910003] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 23:00:43 rpi5 kernel: [34689.912766] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 23:00:43 rpi5 kernel: [34689.915602] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 23:00:43 rpi5 kernel: [34689.918514] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 23:00:43 rpi5 kernel: [34689.921530] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 23:00:43 rpi5 kernel: [34689.924555] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 23:00:43 rpi5 kernel: [34689.927643] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 23:00:43 rpi5 kernel: [34689.930750] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 23:00:43 rpi5 kernel: [34689.933844] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 23:00:43 rpi5 kernel: [34689.936938] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 23:00:43 rpi5 kernel: [34689.940043] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 23:00:43 rpi5 kernel: [34689.943125] Exception stack(0xb9edddb8 to 0xb9edde00)
Mar 30 23:00:43 rpi5 kernel: [34689.944653] dda0: bb3c3400 ba443c94
Mar 30 23:00:43 rpi5 kernel: [34689.947594] ddc0: bb3c3410 bb3c3400 00000001 00000000 00000000 b97c9a60 80c045e8 00000000
Mar 30 23:00:43 rpi5 kernel: [34689.950491] dde0: 80c8826c b9edde2c 00000000 b9edde08 ba443c80 8021d0cc 60000013 ffffffff
Mar 30 23:00:43 rpi5 kernel: [34689.953383] [<8079f6fc>] (__irq_svc) from [<8021d0cc>] (find_get_entry+0x24/0x110)
Mar 30 23:00:43 rpi5 kernel: [34689.956267] [<8021d0cc>] (find_get_entry) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 23:00:43 rpi5 kernel: [34689.959154] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 23:00:43 rpi5 kernel: [34689.962052] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 23:00:43 rpi5 kernel: [34689.964955] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 23:00:43 rpi5 kernel: [34689.967845] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 23:00:43 rpi5 kernel: [34689.969325] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 23:00:43 rpi5 kernel: [34689.970784] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)
Mar 30 23:01:46 rpi5 kernel: [34752.911909] INFO: rcu_sched self-detected stall on CPU
Mar 30 23:01:46 rpi5 kernel: [34752.913370] 2-...: (52459 ticks this GP) idle=7e2/140000000000001/0 softirq=217394/217394 fqs=26214
Mar 30 23:01:46 rpi5 kernel: [34752.916282] (t=52524 jiffies g=139900 c=139899 q=6341)
Mar 30 23:01:46 rpi5 kernel: [34752.917735] NMI backtrace for cpu 2
Mar 30 23:01:46 rpi5 kernel: [34752.919138] CPU: 2 PID: 7937 Comm: exim4 Tainted: G D C 4.14.31-v7+ #1104
Mar 30 23:01:46 rpi5 kernel: [34752.921880] Hardware name: BCM2835
Mar 30 23:01:46 rpi5 kernel: [34752.923219] [<8010fff8>] (unwind_backtrace) from [<8010c260>] (show_stack+0x20/0x24)
Mar 30 23:01:46 rpi5 kernel: [34752.925866] [<8010c260>] (show_stack) from [<80783ac4>] (dump_stack+0xd4/0x118)
Mar 30 23:01:46 rpi5 kernel: [34752.927235] [<80783ac4>] (dump_stack) from [<80789984>] (nmi_cpu_backtrace+0x124/0x128)
Mar 30 23:01:46 rpi5 kernel: [34752.929818] [<80789984>] (nmi_cpu_backtrace) from [<80789a68>] (nmi_trigger_cpumask_backtrace+0xe0/0x120)
Mar 30 23:01:46 rpi5 kernel: [34752.932415] [<80789a68>] (nmi_trigger_cpumask_backtrace) from [<8010e63c>] (arch_trigger_cpumask_backtrace+0x20/0x24)
Mar 30 23:01:46 rpi5 kernel: [34752.935028] [<8010e63c>] (arch_trigger_cpumask_backtrace) from [<80185950>] (rcu_dump_cpu_stacks+0xa8/0xd8)
Mar 30 23:01:46 rpi5 kernel: [34752.937702] [<80185950>] (rcu_dump_cpu_stacks) from [<801851bc>] (rcu_check_callbacks+0x7a4/0x934)
Mar 30 23:01:46 rpi5 kernel: [34752.940417] [<801851bc>] (rcu_check_callbacks) from [<8018b4dc>] (update_process_times+0x40/0x6c)
Mar 30 23:01:46 rpi5 kernel: [34752.943181] [<8018b4dc>] (update_process_times) from [<8019d0bc>] (tick_sched_handle+0x64/0x70)
Mar 30 23:01:46 rpi5 kernel: [34752.946017] [<8019d0bc>] (tick_sched_handle) from [<8019d2fc>] (tick_sched_timer+0x50/0xac)
Mar 30 23:01:46 rpi5 kernel: [34752.948929] [<8019d2fc>] (tick_sched_timer) from [<8018c448>] (__hrtimer_run_queues+0x154/0x2f0)
Mar 30 23:01:46 rpi5 kernel: [34752.951942] [<8018c448>] (__hrtimer_run_queues) from [<8018c820>] (hrtimer_interrupt+0xcc/0x220)
Mar 30 23:01:46 rpi5 kernel: [34752.954961] [<8018c820>] (hrtimer_interrupt) from [<80639710>] (arch_timer_handler_phys+0x40/0x48)
Mar 30 23:01:46 rpi5 kernel: [34752.958050] [<80639710>] (arch_timer_handler_phys) from [<8017a4ec>] (handle_percpu_devid_irq+0x88/0x23c)
Mar 30 23:01:46 rpi5 kernel: [34752.961155] [<8017a4ec>] (handle_percpu_devid_irq) from [<80174f10>] (generic_handle_irq+0x34/0x44)
Mar 30 23:01:46 rpi5 kernel: [34752.964251] [<80174f10>] (generic_handle_irq) from [<80175550>] (__handle_domain_irq+0x6c/0xc4)
Mar 30 23:01:46 rpi5 kernel: [34752.967343] [<80175550>] (__handle_domain_irq) from [<80101504>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Mar 30 23:01:46 rpi5 kernel: [34752.970445] [<80101504>] (bcm2836_arm_irqchip_handle_irq) from [<8079f6fc>] (__irq_svc+0x5c/0x7c)
Mar 30 23:01:46 rpi5 kernel: [34752.973522] Exception stack(0xb9eddd80 to 0xb9edddc8)
Mar 30 23:01:46 rpi5 kernel: [34752.975055] dd80: b97c9a60 00000000 00000000 b9edddf4 ba443c81 00000040 ba443c81 b97c9a60
Mar 30 23:01:46 rpi5 kernel: [34752.978072] dda0: 80c045e8 00000000 80c8826c b9edddec ba443c90 b9edddd0 ba443c80 8078b6a0
Mar 30 23:01:46 rpi5 kernel: [34752.981084] ddc0: a0000013 ffffffff
Mar 30 23:01:46 rpi5 kernel: [34752.982552] [<8079f6fc>] (__irq_svc) from [<8078b6a0>] (__radix_tree_lookup+0x68/0xb4)
Mar 30 23:01:46 rpi5 kernel: [34752.985470] [<8078b6a0>] (__radix_tree_lookup) from [<8078b708>] (radix_tree_lookup_slot+0x1c/0x2c)
Mar 30 23:01:46 rpi5 kernel: [34752.988394] [<8078b708>] (radix_tree_lookup_slot) from [<8021d0d8>] (find_get_entry+0x30/0x110)
Mar 30 23:01:46 rpi5 kernel: [34752.991313] [<8021d0d8>] (find_get_entry) from [<8021d2c8>] (pagecache_get_page+0x44/0x2d0)
Mar 30 23:01:46 rpi5 kernel: [34752.994237] [<8021d2c8>] (pagecache_get_page) from [<80220090>] (generic_file_read_iter+0x200/0x9d0)
Mar 30 23:01:46 rpi5 kernel: [34752.997172] [<80220090>] (generic_file_read_iter) from [<80327bc4>] (ext4_file_read_iter+0x44/0x5c)
Mar 30 23:01:46 rpi5 kernel: [34753.000106] [<80327bc4>] (ext4_file_read_iter) from [<8028a6cc>] (__vfs_read+0xe0/0x130)
Mar 30 23:01:46 rpi5 kernel: [34753.003028] [<8028a6cc>] (__vfs_read) from [<8028a7b8>] (vfs_read+0x9c/0x168)
Mar 30 23:01:46 rpi5 kernel: [34753.004543] [<8028a7b8>] (vfs_read) from [<8028ad68>] (SyS_read+0x54/0xb0)
Mar 30 23:01:46 rpi5 kernel: [34753.006047] [<8028ad68>] (SyS_read) from [<80108080>] (ret_fast_syscall+0x0/0x28)I have a USB harddrive connected to a powered hub. The Pi is connected via wifi only.
ghost
commented
6 years ago +1 I'm seeing this as well. I'm running stock Raspbian Stretch fully up-to-date. Just updated kernel using rpi-update so hopefully that will have fixed it - will report back.
ghost
commented
6 years ago @pelwell Is the fix for this bug the same as for #2446? If so, I'm guessing this bug is a duplicate.
pelwell
commented
6 years ago I think this is a different issue - the TCP bug caused failures in the TCP stack, whereas this looks like the result of a corruption of some kind (memory or filing system).
ghost
commented
6 years ago @pelwell
A patch for that bug was applied to our current branch this afternoon. The latest firmware build, available via rpi-update, includes the patch.
So is that https://github.com/raspberrypi/linux/commit/661230038a8736ba9023978c53cd21cd65739406?
pelwell
commented
6 years ago This thread has been muddled by discussing the TCP bug (#2446), for which the patch is https://github.com/raspberrypi/linux/commit/661230038a8736ba9023978c53cd21cd65739406 as you say.
@comdata's issue appears within the USB stack, and @horgh's is observed within the filing system. They could be caused by the same underlying problem, but at the moment it's too soon to tell.
comdata
commented
6 years ago I could add that the system is running as NAS with cable based ethernet connection and multiple hard discs connected via USB running btrfs on top of it.
Moved back to 4.9.80 and reduced the clock speed to 1200 and it seems stable for now.
ghost
commented
6 years ago @pelwell So to try and unmuddle things: this bug is unrelated to the TCP bug (#2446) and has a different, and as yet unknown cause - is that right?
(I thought the fix for the TCP bug (#2446) was https://github.com/raspberrypi/linux/commit/9d2ad143e40c38d34be86578840499a976c0a5b0?
https://github.com/raspberrypi/linux/commit/661230038a8736ba9023978c53cd21cd65739406 looks different)
pelwell
commented
6 years ago @pelwell So to try and unmuddle things: this bug is unrelated to the TCP bug (#2446) and has a different, and as yet unknown cause - is that right?
Yes.
(I thought the fix for the TCP bug (#2446) was 9d2ad14? 6612300 looks different)
Sigh - too many tabs open. https://github.com/raspberrypi/linux/commit/661230038a8736ba9023978c53cd21cd65739406 is my temporary ethernet driver patch. A complete fix, including pre-open phy access, is proving to be elusively timing dependent, which may explain why the author of the original patch (V2 of which has now been submitted) didn't see a problem.
comdata
commented
6 years ago It seems better with 4.14.32 and arm_freq=1200 set in the config.txt.
Without the arm_freq set it crashes again.
bcutter
commented
6 years ago My system also suffers these kernel crashes, they either start with
Internal error: Oops: 5 [#1] SMP ARM or
Internal error: Oops - undefined instruction: 0 [#1] SMP ARM.
After a certain time watchdog kicks in and reboots the system (Raspbian Stretch):
Message from watchdog:
The system will be rebooted because of error 253 = 'load average too high'ItĀ“s on a brand new Pi 3 B+ (Linux 4.14.30-v7+ #1102 SMP Mon Mar 26 16:45:49 BST 2018 armv7l GNU/Linux), didnĀ“t have that on the same SD card with Raspbian Jessie on a Pi 2 B.
Power voltage should be good (5V 3A), overclocking not available for Pi 3 B+, SD card might be - will rule this out by switching the sd card back into the previous Pi 2 B.
Update:
--> IĀ“m wondering if itĀ“s a hardware or software issue, but definitely something with the Pi 3 B+. Sending it back, trying with another one? Waiting for software updates?
Anything else I can test / provide the experts here?
pelwell
commented
6 years ago Please try with sdram_freq=450 in config.txt. Don't bother putting the same card in a 2B - this doesn't sound like a faulty card.
bcutter
commented
6 years ago THX, found that information here a few minutes ago and set "arm_freq=1200" & "sdram_freq=450", will now switch hardware back to 3 B+ and take a close look.
You ruled out it must be the sdram_freq, so "arm_freq=1200" is not necessary?
pelwell
commented
6 years ago You ruled out it must be the sdram_freq, so "arm_freq=1200" is not necessary?
Yes, for now, based on user feedback. If it turns out that reducing sdram_freq doesn't help in your case then we've learnt something that reducing them both together wouldn't have told us.
bcutter
commented
6 years ago Okay, after some really stable hours Iāll switch now to only cut sdram frequency.
I donāt know whatās the usual sdram freq (āvcgencmd get_config intā should show), but what we buy here for stability is performance. I donāt know how much, but... is it a kernel/Software thing which can be fixed or is it a hardware issue, so not all Pi 3 B + suffer this?
Depending on this Iām thinking of sending my current hardware back, trying my luck with a new/other one...
popcornmix
commented
6 years ago The non-turbo sdram frequency is 400MHz on all devices. On Pi3, the turbo sdram frequency is 450MHz On Pi3 B+, the turbo sdram frequency is 500MHz It would be very useful to determine if this frequency is causing unreliability on your Pi. There are other settings that can be set in software that could help (voltage, delays, schmoo) but we need to know if sdram is actually the problem.
May also be worth running memtester (sudo apt-get install memtester). That tends to be a very effective way of testing if the sdram is not reliable at a given frequency.
bcutter
commented
6 years ago Well, even itĀ“s just a temperature benchmark, the sdram frequency (at the end resulting in temperature, right?) seems to have no direct impact to the performance (core frequency of course has (a huge one)):
System running smooth with only set sdram freq to 450 MHz, core back on default (1.4 GHz).
Ferroin
commented
6 years ago As a general rule, RAM speed will have near zero impact on the temperature reported by the thermal sensor because the thermal sensor is in the CPU, not the RAM. Additionally though, 50MHz isn't likely to make much of a difference in terms of temperature difference even if you were looking at the RAM.
There's unfortunately no particularly quick and easy method to test memory performance on a Pi. A quick and dirty option (but not very accurate) is to create a tmpfs mount, put a moderately large file on it (128M seems to work well) and then use dd with a 4k block size (to match the page size) to copy that to another location on the tmpfs mount and see how long it takes. This involves at least two memcpy() operations for each block, and therefore should do a decent job of providing comparison points for the memory performance. Even here though, I doubt that 50MHz will make much difference.
bcutter
commented
6 years ago Update: already had my first (watchdog initiated) reboot since changed config to only lower sdram frequency.
Message from watchdog: The system will be rebooted because of error 253 = 'load average too high'
This is ātheā typical behavior. Seems like 1.4 GHz is too much.
Ok speaking clear: I spent something between 11 to 13 hours during the last week for investigating and fixing (e. g. almost 10 external fsckās putting the SD card to another pi) all the issues coming from this unstable system on the Pi 3 B+. Iām therefore very frustrated (first time in 5 years of Pi experience) and I think Iāll return that piece of faulty Pi hardware and get another one. I want stability AND performance - and spend my time for more important things -.-
JamesH65
commented
6 years ago @bcutter Would it be possible for you to send the Pi showing these symptoms to us? We would send you back a new 3B+ plus some goodies to cover the return postage? That way we could do the testing inhouse and save you some time. Please send email to info@raspberrypi.org with a FAO to James Hughes so its gets to me. Thanks.
bcutter
commented
6 years ago @JamesH65 : sounds good, basically. Shipping costs and times might be a barrier (Germany - UK; with 9 ā¬ itĀ“s a fourth of the Pi 3 B+ value)... but first thank you for this offer, IĀ“ll think about it.
JamesH65
commented
6 years ago We'll send you something on return that compensates for the postage costs. 9 ā¬ is insignificant to us to try and help figure out what the problem is, but I want to ensure you are not out of pocket.
uglymagoo
commented
6 years ago Is there something new to report? Many Plex users have the some problem with a Raspberry Pi 3 Model B (not+). A downgrade to 4.9.x immediately fixes this problem. Has 4.14.x been pushed to the official raspbian repo? Is is possible to remove this kernel upgrade form the official repos till this issue has been fixed?
pelwell
commented
6 years ago This issue is specific to the 3B+. If you are seeing problems on a 3B then create another issue, including the whole of any stack traces you have, a description of your setup (hardware and software) and what the Pi was doing when it crashed.
pelwell
commented
6 years ago @bcutter We would really like to get our hands on your 3B+. Email me - phil@raspberrypi.org - and I'll explain the deal.
comdata
commented
6 years ago @pelwell if it helps I could send you my B+ I can revert to a 3B till I get a replacement
pelwell
commented
6 years ago Sure - use the email above.
bcutter
commented
6 years ago @pelwell: Since today Iām fighting with what I expected in the first place: heavy SD card corruption. My Piās eat them regularly, about one card per year (current one: 9 months...) - whichās is pretty cost intensive meanwhile running on 128 GB SDXCs. I think Iāll outsource data storage and swap file to USB flash drives in the near future to minimize read/write access for the SD card.
So at the moment Iām restoring from a backup to a fresh SD card. After that I need to retest the Pi 3B+ Hardware with that healthy SD card (donāt want you to send a proper working Pi 3 B+ without very good reasons), at the moment I can not ensure itās the SD card causing all my troubles, even it started at the time when upgrading from Jessie to Stretch and switching to Pi 3 B+. Iāll come back to you in the next days.
To fasten my testing process: thereās a guide on how to stress the memory which I didnāt try yet in the raspberrypi forums. I also know how to stress test CPU and storage, if I find time I will try that with different config.txt settings and check system stability during all runs - I think thatās what youād also do in the first place. And the best cross check test is to run all that tests with my stable Pi 2 B of course.
bcutter
commented
6 years ago SD card now is fine. Tried several as ram overvoltage settings, nothing worked. Only stable setting is armfreq=1200 with sdram freq=450. Will send it back, itās just unacceptable.
JamesH65
commented
6 years ago Sounds like you have an outlying outlier SoC (worst case). Best thing it return it to the supplier, unless you can wait for the SW fix which might take a few days.
bcutter
commented
6 years ago WonĀ“t a software fix slow down the Pi? I donĀ“t understand how it could be fixed, if all that under (frequency) and overclocking (overvoltage) had no positive impact? IĀ“m in contact with dom/Phil already. IĀ“m willing to send mine to anyone of you if it helps to diagnose/analyse. I only want one thing: a powerful and stable Pi 3 B+ :-)
pelwell
commented
6 years ago The link between the SoC and the SDRAM has a number of adjustable parameters, and the hope is that with the right settings there will be no need to reduce the clocks.
JamesH65
commented
6 years ago @comdata Are you and any others still seeing the USB issue? I suspect this was a 3B+ SoC/SDRAM timing issue rather than anything related to the USB.
comdata
commented
6 years ago no the issue seems to be related to the specific unit. with the replacement unit I can not see this issue and it survived the stress I put on it so far.
I would suggest to have the new unit running for week and if it doesn't occur we can close the ticket.
On Mon, Apr 23, 2018 at 2:47 PM, James Hughes notifications@github.com wrote:
@comdata https://github.com/comdata Are you and any others still seeing the USB issue? I suspect this was a 3B+ SoC/SDRAM timing issue rather than anything related to the USB.
ā You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raspberrypi/linux/issues/2481#issuecomment-383562527, or mute the thread https://github.com/notifications/unsubscribe-auth/AEe6z6sKKTLbJ-AOb1aGqcLoyr1rvwACks5trc1WgaJpZM4TAe8C .
comdata
commented
6 years ago haven't seen this issue on the replacement units so will close this. Thanks everyone.
JamesH65
commented
6 years ago Great, thanks for the update, glad it's all working now.
Hello,
I am running a PI 3B+ with hard disks attached via USB. All hard disks are self powered. When running with 4.14.30 v7 I am getting regularly kernel oops under high load (e.g. compiling something)
Here is a screenshot:
Reverting back to 4.9.80 seems to fix the issue.
Kind Regards