Joshua-Riek / ubuntu-rockchip

Ubuntu for Rockchip RK35XX Devices
https://joshua-riek.github.io/ubuntu-rockchip-download/
GNU General Public License v3.0
2.53k stars 274 forks source link

Bug Report: EMMC CQE Recovery resulting in read-only filesystem #1060

Open tomharvey opened 2 months ago

tomharvey commented 2 months ago

What happened?

I was doing some intensive disk activity on my EMMC device (no other disks are connected) when it flipped to read-only mode.

Kernel logging is attached

This seems similar to issues which resulted in a reconfiguration of the EMMC in the official kernel here https://github.com/radxa/kernel/pull/127/files and issues reported in Armbian here https://forum.armbian.com/topic/18745-emmc-errors/

Kernel version

6.1.0-1025-rockchip

SBC model

Radxa Rock 5B

What operating system are you seeing this problem on?

Ubuntu 24.04 LTS (Noble Nombat)

Relevant logs

2024-09-26T11:32:35.548496+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:36.067121+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:36.067167+00:00 ubuntu kernel: ------------[ cut here ]------------
2024-09-26T11:32:36.067172+00:00 ubuntu kernel: mmc0: cqhci: spurious TCN for tag 9
2024-09-26T11:32:36.067182+00:00 ubuntu kernel: WARNING: CPU: 0 PID: 6139 at drivers/mmc/host/cqhci-core.c:791 cqhci_irq+0x260/0x2e8
2024-09-26T11:32:36.067184+00:00 ubuntu kernel: Modules linked in: veth xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat bridge stp llc nf_conntrack_netlink nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype nft_compat nf_tables overlay btusb btmtk qrtr bnep binfmt_misc pwm_fan hci_uart rtw89_8852be btqca rtw89_8852b rtw89_pci rtw89_core rtk_btusb panfrost btrtl btbcm btintel drm_shmem_helper gpu_sched input_leds joydev uio_pdrv_genirq uio squashfs dm_multipath fuse dm_mod nfnetlink ip_tables ipv6 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx raid1 raid0 multipath linear md_mod r8169 [last unloaded: btmtk]
2024-09-26T11:32:36.067186+00:00 ubuntu kernel: CPU: 0 PID: 6139 Comm: kworker/0:1H Not tainted 6.1.0-1025-rockchip #25-Ubuntu
2024-09-26T11:32:36.067188+00:00 ubuntu kernel: Hardware name: Radxa ROCK 5B (DT)
2024-09-26T11:32:36.067189+00:00 ubuntu kernel: Workqueue: kblockd blk_mq_run_work_fn
2024-09-26T11:32:36.067192+00:00 ubuntu kernel: pstate: 604000c9 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
2024-09-26T11:32:36.067195+00:00 ubuntu kernel: pc : cqhci_irq+0x260/0x2e8
2024-09-26T11:32:36.067197+00:00 ubuntu kernel: lr : cqhci_irq+0x260/0x2e8
2024-09-26T11:32:36.067199+00:00 ubuntu kernel: sp : ffff800008003db0
2024-09-26T11:32:36.067200+00:00 ubuntu kernel: x29: ffff800008003dc0 x28: 0000000003ff003f x27: 000000000270003e
2024-09-26T11:32:36.067202+00:00 ubuntu kernel: x26: 00000000010f0001 x25: ffff0001003a9698 x24: ffff80000b298eef
2024-09-26T11:32:36.067203+00:00 ubuntu kernel: x23: ffff80000b2e6160 x22: 0000000000000009 x21: 0000000000000002
2024-09-26T11:32:36.067205+00:00 ubuntu kernel: x20: ffff0001003a9680 x19: ffff000100245000 x18: 0000000000000000
2024-09-26T11:32:36.067206+00:00 ubuntu kernel: x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
2024-09-26T11:32:36.067210+00:00 ubuntu kernel: x14: 0000000000000000 x13: 392067617420726f x12: 66204e4354207375
2024-09-26T11:32:36.067213+00:00 ubuntu kernel: x11: 6f6972757073203a x10: 6963687163203a30 x9 : ffff8000080ea040
2024-09-26T11:32:36.067214+00:00 ubuntu kernel: x8 : 43542073756f6972 x7 : 757073203a696368 x6 : 7163203a30636d6d
2024-09-26T11:32:36.067216+00:00 ubuntu kernel: x5 : ffff0004fdda9a90 x4 : 0000000000000000 x3 : 0000000000000027
2024-09-26T11:32:36.067218+00:00 ubuntu kernel: x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff000104cacec0
2024-09-26T11:32:36.067220+00:00 ubuntu kernel: Call trace:
2024-09-26T11:32:36.067222+00:00 ubuntu kernel:  cqhci_irq+0x260/0x2e8
2024-09-26T11:32:36.067224+00:00 ubuntu kernel:  dwcmshc_cqe_irq_handler+0x5c/0x90
2024-09-26T11:32:36.067226+00:00 ubuntu kernel:  sdhci_irq+0x260/0x3d0
2024-09-26T11:32:36.067227+00:00 ubuntu kernel:  __handle_irq_event_percpu+0xbc/0x1c0
2024-09-26T11:32:36.067229+00:00 ubuntu kernel:  handle_irq_event_percpu+0x20/0x54
2024-09-26T11:32:36.067231+00:00 ubuntu kernel:  handle_irq_event+0x50/0x94
2024-09-26T11:32:36.067233+00:00 ubuntu kernel:  handle_fasteoi_irq+0xa4/0x124
2024-09-26T11:32:36.067234+00:00 ubuntu kernel:  handle_irq_desc+0x28/0x40
2024-09-26T11:32:36.067236+00:00 ubuntu kernel:  generic_handle_domain_irq+0x20/0x2c
2024-09-26T11:32:36.067239+00:00 ubuntu kernel:  __gic_handle_irq_from_irqson.isra.0+0x140/0x1c4
2024-09-26T11:32:36.067240+00:00 ubuntu kernel:  gic_handle_irq+0x88/0x90
2024-09-26T11:32:36.067242+00:00 ubuntu kernel:  call_on_irq_stack+0x24/0x4c
2024-09-26T11:32:36.067244+00:00 ubuntu kernel:  do_interrupt_handler+0x64/0xa8
2024-09-26T11:32:36.067245+00:00 ubuntu kernel:  el1_interrupt+0x90/0xa0
2024-09-26T11:32:36.067247+00:00 ubuntu kernel:  el1h_64_irq_handler+0x18/0x24
2024-09-26T11:32:36.067249+00:00 ubuntu kernel:  el1h_64_irq+0x74/0x78
2024-09-26T11:32:36.067250+00:00 ubuntu kernel:  sdhci_cqe_enable+0x130/0x150
2024-09-26T11:32:36.067253+00:00 ubuntu kernel:  cqhci_request+0x148/0x460
2024-09-26T11:32:36.067255+00:00 ubuntu kernel:  mmc_cqe_start_req+0x6c/0x104
2024-09-26T11:32:36.067256+00:00 ubuntu kernel:  mmc_blk_mq_issue_rq+0x234/0x3ac
2024-09-26T11:32:36.067258+00:00 ubuntu kernel:  mmc_mq_queue_rq+0x19c/0x21c
2024-09-26T11:32:36.067260+00:00 ubuntu kernel:  blk_mq_dispatch_rq_list+0x234/0x39c
2024-09-26T11:32:36.067262+00:00 ubuntu kernel:  __blk_mq_sched_dispatch_requests+0xc0/0x14c
2024-09-26T11:32:36.067264+00:00 ubuntu kernel:  blk_mq_sched_dispatch_requests+0x40/0x78
2024-09-26T11:32:36.067265+00:00 ubuntu kernel:  __blk_mq_run_hw_queue+0x70/0x84
2024-09-26T11:32:36.067385+00:00 ubuntu kernel:  blk_mq_run_work_fn+0x30/0x40
2024-09-26T11:32:36.067394+00:00 ubuntu kernel:  process_one_work+0x1a0/0x250
2024-09-26T11:32:36.067395+00:00 ubuntu kernel:  worker_thread+0x1dc/0x248
2024-09-26T11:32:36.067396+00:00 ubuntu kernel:  kthread+0xc0/0xd0
2024-09-26T11:32:36.067398+00:00 ubuntu kernel:  ret_from_fork+0x10/0x20
2024-09-26T11:32:36.067399+00:00 ubuntu kernel: 
2024-09-26T11:32:36.067401+00:00 ubuntu kernel: PC: 0xffff800008d8d640:
2024-09-26T11:32:36.067404+00:00 ubuntu kernel: d440  2a0003e1 2a0003f5 aa1403e0 97fffe90 3941cee0 36000120 f9402e62 b5000042
2024-09-26T11:32:36.067406+00:00 ubuntu kernel: d460  f9400662 f0008681 2a1503e3 91180021 910142e0 97e431ec 52800680 6a0002bf
2024-09-26T11:32:36.067407+00:00 ubuntu kernel: d480  54000861 2a1802c0 350009a0 360805b5 52800581 aa1403e0 97fffe4a 2a0003e1
2024-09-26T11:32:36.067408+00:00 ubuntu kernel: d4a0  2a0003e0 52800582 f90003e0 aa1403e0 97fffe77 39426ee0 36000120 f9402e62
2024-09-26T11:32:36.067410+00:00 ubuntu kernel: d4c0  b5000042 f9400662 f0008681 f94003e3 91187c21 9101e2e0 97e431d3 91006299
2024-09-26T11:32:36.067411+00:00 ubuntu kernel: d4e0  f0012858 aa1903e0 913bbf18 941358a3 d2800002 b9803a81 910003e0 97e2f5a5
2024-09-26T11:32:36.067413+00:00 ubuntu kernel: d500  aa0003f6 b9803a80 eb16001f 54000648 39414280 36000120 b9403e80 350000e0
2024-09-26T11:32:36.067414+00:00 ubuntu kernel: d520  3901429f 91032280 d2800003 52800022 52800061 97cd2c5f aa1903e0 941358c2
2024-09-26T11:32:36.067416+00:00 ubuntu kernel: d540  361800d5 91032280 d2800003 52800022 52800061 97cd2c57 360000d5 91032280
2024-09-26T11:32:36.067418+00:00 ubuntu kernel: d560  d2800003 52800022 52800061 97cd2c51 d5384100 f94007e2 f9429c01 eb010042
2024-09-26T11:32:36.067419+00:00 ubuntu kernel: d580  d2800001 54000900 9413404f 36100095 b945c660 11000400 b905c660 36200095
2024-09-26T11:32:36.067420+00:00 ubuntu kernel: d5a0  b945ca60 11000400 b905ca60 36280095 b945ce60 11000400 b905ce60 2a1803e3
2024-09-26T11:32:36.067422+00:00 ubuntu kernel: d5c0  2a1603e2 2a1503e1 aa1303e0 97fffee8 17ffffaf 39430ee0 36000120 f9402e62
2024-09-26T11:32:36.067423+00:00 ubuntu kernel: d5e0  b5000042 f9400662 f0008681 aa1603e3 9118e021 910282e0 97e4318b f942ce60
2024-09-26T11:32:36.067425+00:00 ubuntu kernel: d600  d37c7ec2 f9407803 8b020064 f8626861 b50001e1 39400b00 37000160 f9402e61
2024-09-26T11:32:36.067426+00:00 ubuntu kernel: d620  52800020 39000b00 b5000041 f9400661 2a1603e2 f0008680 91195c00 97cbdda0
2024-09-26T11:32:36.067427+00:00 ubuntu kernel: d640* d4210000 910006c2 17ffffab 39414405 360000a5 b9400880 321f0000 b9000880
2024-09-26T11:32:36.067428+00:00 ubuntu kernel: d660  17fffff9 f822687f b9403c02 51000442 b9003c02 f9400820 b4000080 b9401402
2024-09-26T11:32:36.067429+00:00 ubuntu kernel: d680  340000a2 b9001c1f aa1303e0 97ff5377 17ffffed 29410c02 1b037c42 b9001c02
2024-09-26T11:32:36.067432+00:00 ubuntu kernel: d6a0  17fffffa a9417bfd 52800020 a94253f3 a9435bf5 a94463f7 f9402bf9 910183ff
2024-09-26T11:32:36.067433+00:00 ubuntu kernel: d6c0  d50323bf d65f03c0 aa1e03e9 d503201f d503233f a9bd7bfd f0008682 910003fd
2024-09-26T11:32:36.067434+00:00 ubuntu kernel: d6e0  9119ec42 52804001 a90153f3 aa0003f3 f90013f5 97efe8ca b5000180 b0012ac0
2024-09-26T11:32:36.067436+00:00 ubuntu kernel: d700  91058000 91032000 39408c01 360000a1 f0008682 91004261 911a0442 97e43176
2024-09-26T11:32:36.067437+00:00 ubuntu kernel: d720  928002b4 14000015 91004273 aa0003f5 5281b802 aa1303e0 d2801f01 97eff285
2024-09-26T11:32:36.067439+00:00 ubuntu kernel: d740  aa0003f4 b40003a0 a9400aa1 aa1303e0 91000442 cb010042 97e3255f f9000680
2024-09-26T11:32:36.067440+00:00 ubuntu kernel: d760  b5000180 928001f4 aa1303e0 f0008681 911a5421 94125790 f94013f5 aa1403e0
2024-09-26T11:32:36.067441+00:00 ubuntu kernel: d780  a94153f3 a8c37bfd d50323bf d65f03c0 b0012ac0 91058000 9103c000 39408c01
2024-09-26T11:32:36.067443+00:00 ubuntu kernel: d7a0  3607fec1 aa1303e1 f0008682 911ac442 97e43151 17fffff1 92800174 17ffffef
2024-09-26T11:32:36.067444+00:00 ubuntu kernel: d7c0  aa1e03e9 d503201f d503233f a9ba7bfd 910003fd a90153f3 aa0003f4 aa0103f3
2024-09-26T11:32:36.067446+00:00 ubuntu kernel: d7e0  a9025bf5 a90363f7 aa0203f8 a9046bf9 a90573fb f942cc15 f9400420 b50005c0
2024-09-26T11:32:36.067448+00:00 ubuntu kernel: d800  b9409c36 f9407aba 937c7ed9 910062bb aa1b03e0 941357f0 f8796b59 8b36d35c
2024-09-26T11:32:36.067449+00:00 ubuntu kernel: d820  aa0003f7 eb13033f 1a9f17fa 54000141 b9400b80 aa1303e1 52800002 32000000
2024-09-26T11:32:36.067451+00:00 ubuntu kernel: 
2024-09-26T11:32:36.067452+00:00 ubuntu kernel: LR: 0xffff800008d8d640:
2024-09-26T11:32:36.067454+00:00 ubuntu kernel: d440  2a0003e1 2a0003f5 aa1403e0 97fffe90 3941cee0 36000120 f9402e62 b5000042
2024-09-26T11:32:36.067455+00:00 ubuntu kernel: d460  f9400662 f0008681 2a1503e3 91180021 910142e0 97e431ec 52800680 6a0002bf
2024-09-26T11:32:36.067458+00:00 ubuntu kernel: d480  54000861 2a1802c0 350009a0 360805b5 52800581 aa1403e0 97fffe4a 2a0003e1
2024-09-26T11:32:36.067459+00:00 ubuntu kernel: d4a0  2a0003e0 52800582 f90003e0 aa1403e0 97fffe77 39426ee0 36000120 f9402e62
2024-09-26T11:32:36.067461+00:00 ubuntu kernel: d4c0  b5000042 f9400662 f0008681 f94003e3 91187c21 9101e2e0 97e431d3 91006299
2024-09-26T11:32:36.067462+00:00 ubuntu kernel: d4e0  f0012858 aa1903e0 913bbf18 941358a3 d2800002 b9803a81 910003e0 97e2f5a5
2024-09-26T11:32:36.067465+00:00 ubuntu kernel: d500  aa0003f6 b9803a80 eb16001f 54000648 39414280 36000120 b9403e80 350000e0
2024-09-26T11:32:36.067467+00:00 ubuntu kernel: d520  3901429f 91032280 d2800003 52800022 52800061 97cd2c5f aa1903e0 941358c2
2024-09-26T11:32:36.067468+00:00 ubuntu kernel: d540  361800d5 91032280 d2800003 52800022 52800061 97cd2c57 360000d5 91032280
2024-09-26T11:32:36.067469+00:00 ubuntu kernel: d560  d2800003 52800022 52800061 97cd2c51 d5384100 f94007e2 f9429c01 eb010042
2024-09-26T11:32:36.067471+00:00 ubuntu kernel: d580  d2800001 54000900 9413404f 36100095 b945c660 11000400 b905c660 36200095
2024-09-26T11:32:36.067473+00:00 ubuntu kernel: d5a0  b945ca60 11000400 b905ca60 36280095 b945ce60 11000400 b905ce60 2a1803e3
2024-09-26T11:32:36.067476+00:00 ubuntu kernel: d5c0  2a1603e2 2a1503e1 aa1303e0 97fffee8 17ffffaf 39430ee0 36000120 f9402e62
2024-09-26T11:32:36.067557+00:00 ubuntu kernel: d5e0  b5000042 f9400662 f0008681 aa1603e3 9118e021 910282e0 97e4318b f942ce60
2024-09-26T11:32:36.067558+00:00 ubuntu kernel: d600  d37c7ec2 f9407803 8b020064 f8626861 b50001e1 39400b00 37000160 f9402e61
2024-09-26T11:32:36.067559+00:00 ubuntu kernel: d620  52800020 39000b00 b5000041 f9400661 2a1603e2 f0008680 91195c00 97cbdda0
2024-09-26T11:32:36.067559+00:00 ubuntu kernel: d640* d4210000 910006c2 17ffffab 39414405 360000a5 b9400880 321f0000 b9000880
2024-09-26T11:32:36.067560+00:00 ubuntu kernel: d660  17fffff9 f822687f b9403c02 51000442 b9003c02 f9400820 b4000080 b9401402
2024-09-26T11:32:36.067561+00:00 ubuntu kernel: d680  340000a2 b9001c1f aa1303e0 97ff5377 17ffffed 29410c02 1b037c42 b9001c02
2024-09-26T11:32:36.067562+00:00 ubuntu kernel: d6a0  17fffffa a9417bfd 52800020 a94253f3 a9435bf5 a94463f7 f9402bf9 910183ff
2024-09-26T11:32:36.067562+00:00 ubuntu kernel: d6c0  d50323bf d65f03c0 aa1e03e9 d503201f d503233f a9bd7bfd f0008682 910003fd
2024-09-26T11:32:36.067563+00:00 ubuntu kernel: d6e0  9119ec42 52804001 a90153f3 aa0003f3 f90013f5 97efe8ca b5000180 b0012ac0
2024-09-26T11:32:36.067564+00:00 ubuntu kernel: d700  91058000 91032000 39408c01 360000a1 f0008682 91004261 911a0442 97e43176
2024-09-26T11:32:36.067564+00:00 ubuntu kernel: d720  928002b4 14000015 91004273 aa0003f5 5281b802 aa1303e0 d2801f01 97eff285
2024-09-26T11:32:36.067565+00:00 ubuntu kernel: d740  aa0003f4 b40003a0 a9400aa1 aa1303e0 91000442 cb010042 97e3255f f9000680
2024-09-26T11:32:36.067566+00:00 ubuntu kernel: d760  b5000180 928001f4 aa1303e0 f0008681 911a5421 94125790 f94013f5 aa1403e0
2024-09-26T11:32:36.067566+00:00 ubuntu kernel: d780  a94153f3 a8c37bfd d50323bf d65f03c0 b0012ac0 91058000 9103c000 39408c01
2024-09-26T11:32:36.067567+00:00 ubuntu kernel: d7a0  3607fec1 aa1303e1 f0008682 911ac442 97e43151 17fffff1 92800174 17ffffef
2024-09-26T11:32:36.067568+00:00 ubuntu kernel: d7c0  aa1e03e9 d503201f d503233f a9ba7bfd 910003fd a90153f3 aa0003f4 aa0103f3
2024-09-26T11:32:36.067569+00:00 ubuntu kernel: d7e0  a9025bf5 a90363f7 aa0203f8 a9046bf9 a90573fb f942cc15 f9400420 b50005c0
2024-09-26T11:32:36.067569+00:00 ubuntu kernel: d800  b9409c36 f9407aba 937c7ed9 910062bb aa1b03e0 941357f0 f8796b59 8b36d35c
2024-09-26T11:32:36.067570+00:00 ubuntu kernel: d820  aa0003f7 eb13033f 1a9f17fa 54000141 b9400b80 aa1303e1 52800002 32000000
2024-09-26T11:32:36.067571+00:00 ubuntu kernel: 
2024-09-26T11:32:36.067571+00:00 ubuntu kernel: SP: 0xffff800008003db0:
2024-09-26T11:32:36.067572+00:00 ubuntu kernel: 3bb0  08003bd0 ffff8000 080327c0 ffff8000 08003c60 ffff8000 ffffffff 00000000
2024-09-26T11:32:36.067573+00:00 ubuntu kernel: 3bd0  08003c20 ffff8000 0925c860 ffff8000 08003c60 ffff8000 0a91e2f4 ffff8000
2024-09-26T11:32:36.067574+00:00 ubuntu kernel: 3bf0  f2000800 00000000 44000080 f7e1fddf 604000c9 00000000 0b298eef ffff8000
2024-09-26T11:32:36.067574+00:00 ubuntu kernel: 3c10  003a9698 ffff0001 010f0001 00000000 08003c50 ffff8000 0925cc80 ffff8000
2024-09-26T11:32:36.067575+00:00 ubuntu kernel: 3c30  00245000 ffff0001 003a9680 ffff0001 08003db0 ffff8000 08d8d640 ffff8000
2024-09-26T11:32:36.067576+00:00 ubuntu kernel: 3c50  08003d90 ffff8000 080112e0 ffff8000 04cacec0 ffff0001 00000000 00000000
2024-09-26T11:32:36.067577+00:00 ubuntu kernel: 3c70  00000000 00000000 00000027 00000000 00000000 00000000 fdda9a90 ffff0004
2024-09-26T11:32:36.067578+00:00 ubuntu kernel: 3c90  30636d6d 7163203a 3a696368 75707320 756f6972 43542073 080ea040 ffff8000
2024-09-26T11:32:36.067579+00:00 ubuntu kernel: 3cb0  63203a30 69636871 7073203a 6f697275 54207375 66204e43 7420726f 39206761
2024-09-26T11:32:36.067579+00:00 ubuntu kernel: 3cd0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067580+00:00 ubuntu kernel: 3cf0  00000000 00000000 00245000 ffff0001 003a9680 ffff0001 00000002 00000000
2024-09-26T11:32:36.067581+00:00 ubuntu kernel: 3d10  00000009 00000000 0b2e6160 ffff8000 0b298eef ffff8000 003a9698 ffff0001
2024-09-26T11:32:36.067582+00:00 ubuntu kernel: 3d30  010f0001 00000000 0270003e 00000000 03ff003f 00000000 08003dc0 ffff8000
2024-09-26T11:32:36.067582+00:00 ubuntu kernel: 3d50  08d8d640 ffff8000 08003db0 ffff8000 08d8d640 ffff8000 604000c9 00000000
2024-09-26T11:32:36.067583+00:00 ubuntu kernel: 3d70  08003da0 ffff8000 0389bd00 ffff0001 00000009 00000000 0097d480 ffff0001
2024-09-26T11:32:36.067584+00:00 ubuntu kernel: 3d90  08003dc0 ffff8000 08d8d640 ffff8000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067585+00:00 ubuntu kernel: 3db0* 00000200 00000000 943f8800 498c6f08 08003e20 ffff8000 08d8bad0 ffff8000
2024-09-26T11:32:36.067586+00:00 ubuntu kernel: 3dd0  00000000 00000000 00245600 ffff0001 003a9400 ffff0001 00000000 00000000
2024-09-26T11:32:36.067586+00:00 ubuntu kernel: 3df0  0b2e5af8 ffff8000 002458f0 ffff0001 00000010 00000000 08d8bab8 ffff8000
2024-09-26T11:32:36.067587+00:00 ubuntu kernel: 3e10  00000000 00000000 943f8800 498c6f08 08003e60 ffff8000 08d839a4 ffff8000
2024-09-26T11:32:36.067588+00:00 ubuntu kernel: 3e30  00245600 ffff0001 00000000 00000000 fdda9f00 00004000 00000000 00000000
2024-09-26T11:32:36.067588+00:00 ubuntu kernel: 3e50  00000000 00000000 943f8800 498c6f08 08003ec0 ffff8000 080ed084 ffff8000
2024-09-26T11:32:36.067589+00:00 ubuntu kernel: 3e70  038a0280 ffff0001 00000061 00000000 003a9400 ffff0001 003a94b0 ffff0001
2024-09-26T11:32:36.067590+00:00 ubuntu kernel: 3e90  00000000 00000000 0b299da0 ffff8000 0b299de0 ffff8000 00000000 00000000
2024-09-26T11:32:36.067591+00:00 ubuntu kernel: 3eb0  00000001 00000000 04cacec0 ffff0001 08003f10 ffff8000 080ed1a8 ffff8000
2024-09-26T11:32:36.067592+00:00 ubuntu kernel: 3ed0  003a9400 ffff0001 0b135b10 ffff8000 003a94b0 ffff0001 003a94b0 ffff0001
2024-09-26T11:32:36.067593+00:00 ubuntu kernel: 3ef0  60400009 00000000 0000000a 00000000 00000001 00000000 00000000 00000000
2024-09-26T11:32:36.067594+00:00 ubuntu kernel: 3f10  08003f30 ffff8000 080ed22c ffff8000 003a9400 ffff0001 0b135b10 ffff8000
2024-09-26T11:32:36.067596+00:00 ubuntu kernel: 3f30  08003f60 ffff8000 080f27d4 ffff8000 003a9400 ffff0001 0b135b10 ffff8000
2024-09-26T11:32:36.067598+00:00 ubuntu kernel: 3f50  00004000 00000000 080f2764 ffff8000 08003fa0 ffff8000 080ec7f4 ffff8000
2024-09-26T11:32:36.067599+00:00 ubuntu kernel: 3f70  000000ed 00000000 00000000 00000000 00000000 00000000 08d7e470 ffff8000
2024-09-26T11:32:36.067600+00:00 ubuntu kernel: 3f90  60400009 00000000 08d5bf38 ffff8000 08003fb0 ffff8000 080ec8a4 ffff8000
2024-09-26T11:32:36.067602+00:00 ubuntu kernel: 
2024-09-26T11:32:36.067603+00:00 ubuntu kernel: X0: 0xffff000104cacec0:
2024-09-26T11:32:36.067605+00:00 ubuntu kernel: ccc0  00000010 00000000 0000afe0 00000040 00000001 00000000 00000000 00000000
2024-09-26T11:32:36.067608+00:00 ubuntu kernel: cce0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067609+00:00 ubuntu kernel: cd00  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067611+00:00 ubuntu kernel: cd20  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067613+00:00 ubuntu kernel: cd40  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067614+00:00 ubuntu kernel: cd60  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067616+00:00 ubuntu kernel: cd80  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067618+00:00 ubuntu kernel: cda0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067619+00:00 ubuntu kernel: cdc0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067621+00:00 ubuntu kernel: cde0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067623+00:00 ubuntu kernel: ce00  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067625+00:00 ubuntu kernel: ce20  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067626+00:00 ubuntu kernel: ce40  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067627+00:00 ubuntu kernel: ce60  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067629+00:00 ubuntu kernel: ce80  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067631+00:00 ubuntu kernel: cea0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067633+00:00 ubuntu kernel: cec0* 00000008 00000000 00010000 00000001 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067634+00:00 ubuntu kernel: cee0  0fa08000 ffff8000 00000001 04208060 00000000 00000001 00000000 00000000
2024-09-26T11:32:36.067636+00:00 ubuntu kernel: cf00  00000030 00000000 00000023 00000000 016cd110 00000001 03004ec0 ffff0001
2024-09-26T11:32:36.067638+00:00 ubuntu kernel: cf20  00000003 00000000 00000001 00000064 00000064 00000064 00000000 00000000
2024-09-26T11:32:36.067667+00:00 ubuntu kernel: cf40  056ae400 00000000 0000bd04 00000000 00000001 00000000 0dc1eed0 ffff0001
2024-09-26T11:32:36.067669+00:00 ubuntu kernel: cf60  00000000 00000000 0dc1eee8 ffff0001 fddba1b8 ffff0004 00000001 00000000
2024-09-26T11:32:36.067670+00:00 ubuntu kernel: cf80  312fcf19 000048c3 3550f20a 00000006 08160a15 00000049 3550f20a 00000006
2024-09-26T11:32:36.067670+00:00 ubuntu kernel: cfa0  00000001 00000000 00000000 00000000 00000000 00000000 fddb97c0 ffff0004
2024-09-26T11:32:36.067671+00:00 ubuntu kernel: cfc0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067672+00:00 ubuntu kernel: cfe0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067672+00:00 ubuntu kernel: d000  2f6a1400 000048c3 00000000 00000000 00000000 00000000 00000000 00000285
2024-09-26T11:32:36.067673+00:00 ubuntu kernel: d020  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000070
2024-09-26T11:32:36.067675+00:00 ubuntu kernel: d040  04cad040 ffff0001 04cad040 ffff0001 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067676+00:00 ubuntu kernel: d060  0000001e 00000000 00000000 00000000 04cad070 ffff0001 00000000 00000000
2024-09-26T11:32:36.067686+00:00 ubuntu kernel: d080  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067687+00:00 ubuntu kernel: d0a0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067745+00:00 ubuntu kernel: 
2024-09-26T11:32:36.067747+00:00 ubuntu kernel: X5: 0xffff0004fdda9a90:
2024-09-26T11:32:36.067747+00:00 ubuntu kernel: 9890  00000000 00000000 00000000 00000000 08771428 0000000b 694d30fa 00000000
2024-09-26T11:32:36.067749+00:00 ubuntu kernel: 98b0  3e20e987 00000015 2d9a732b 00000003 00000000 00000000 d09d7f49 0000489e
2024-09-26T11:32:36.067750+00:00 ubuntu kernel: 98d0  dfb750de 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067751+00:00 ubuntu kernel: 98f0  00207e07 00000000 000006e8 00033b92 00000005 00000544 00014932 00000000
2024-09-26T11:32:36.067753+00:00 ubuntu kernel: 9910  0000096e 00074372 00000000 0001cb66 000000ff 00000000 000000ff 00000000
2024-09-26T11:32:36.067754+00:00 ubuntu kernel: 9930  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067756+00:00 ubuntu kernel: 9950  000000f0 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067758+00:00 ubuntu kernel: 9970  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067760+00:00 ubuntu kernel: 9990  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067761+00:00 ubuntu kernel: 99b0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067763+00:00 ubuntu kernel: 99d0  0444ba00 ffff0001 0444ba00 ffff0001 00000008 00000000 04930a00 ffff0001
2024-09-26T11:32:36.067765+00:00 ubuntu kernel: 99f0  00000000 00000000 00000000 00000000 00475f02 00000000 00000000 00000002
2024-09-26T11:32:36.067766+00:00 ubuntu kernel: 9a10  00000000 000000d0 6b554b5b 35b7f828 00000000 00000000 5d69651c 20f44ee5
2024-09-26T11:32:36.067767+00:00 ubuntu kernel: 9a30  cb1b9bbe 00000000 312fd3a8 000048c3 6b554b5b 35b7f828 00000000 00000000
2024-09-26T11:32:36.067768+00:00 ubuntu kernel: 9a50  5d634e82 20efacda c9df05ff 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067769+00:00 ubuntu kernel: 9a70  00000000 00000000 00000000 00000000 80b624d9 00000023 00000001 00000000
2024-09-26T11:32:36.067770+00:00 ubuntu kernel: 9a90* 00000000 00000000 00000027 00000000 080e92e8 ffff8000 00000000 00000000
2024-09-26T11:32:36.067771+00:00 ubuntu kernel: 9ab0  00000000 00000000 00000000 00000000 00000000 00000000 fdda9ac0 ffff0004
2024-09-26T11:32:36.067771+00:00 ubuntu kernel: 9ad0  fdda9ac0 ffff0004 fdda9ac0 ffff0004 fdda9ac0 ffff0004 00000000 00000000
2024-09-26T11:32:36.067772+00:00 ubuntu kernel: 9af0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067773+00:00 ubuntu kernel: 9b10  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067773+00:00 ubuntu kernel: 9b30  00000001 00000000 00000000 dead4ead ffffffff 00000000 ffffffff ffffffff
2024-09-26T11:32:36.067774+00:00 ubuntu kernel: 9b50  00000000 00000000 00000000 00000000 ffffffe0 0000000f fdda9b68 ffff0004
2024-09-26T11:32:36.067774+00:00 ubuntu kernel: 9b70  fdda9b68 ffff0004 080ffb9c ffff8000 00000000 00000000 00000008 00000000
2024-09-26T11:32:36.067775+00:00 ubuntu kernel: 9b90  080fed10 ffff8000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067775+00:00 ubuntu kernel: 9bb0  fdda9bb0 ffff0004 fdda9bb0 ffff0004 00000000 00000000 0b0769b8 ffff8000
2024-09-26T11:32:36.067776+00:00 ubuntu kernel: 9bd0  00000000 00000000 fdda9bd0 ffff0004 fdda9bd0 ffff0004 fdda9bd0 ffff0004
2024-09-26T11:32:36.067777+00:00 ubuntu kernel: 9bf0  fdda9bd0 ffff0004 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067777+00:00 ubuntu kernel: 9c10  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067778+00:00 ubuntu kernel: 9c30  00000000 00000000 00000000 00000000 00000001 00000000 00000000 dead4ead
2024-09-26T11:32:36.067779+00:00 ubuntu kernel: 9c50  ffffffff 00000000 ffffffff ffffffff 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067779+00:00 ubuntu kernel: 9c70  ffffffe0 0000000f fdda9c78 ffff0004 fdda9c78 ffff0004 080ffb9c ffff8000
2024-09-26T11:32:36.067780+00:00 ubuntu kernel: 
2024-09-26T11:32:36.067781+00:00 ubuntu kernel: X9: 0xffff8000080ea040:
2024-09-26T11:32:36.067781+00:00 ubuntu kernel: 9e40  97fff64d b00192a0 52800021 b905f801 97fff902 a8c17bfd d50323bf d65f03c0
2024-09-26T11:32:36.067782+00:00 ubuntu kernel: 9e60  d65f03c0 aa1e03e9 d503201f 90017561 91322020 39722021 360001a1 d503233f
2024-09-26T11:32:36.067782+00:00 ubuntu kernel: 9e80  a9bf7bfd 91028000 910003fd 9445dac1 b00192a0 b905f81f 97fffc57 97fffd96
2024-09-26T11:32:36.067783+00:00 ubuntu kernel: 9ea0  a8c17bfd d50323bf d65f03c0 d65f03c0 aa1e03e9 d503201f d503233f a9be7bfd
2024-09-26T11:32:36.067784+00:00 ubuntu kernel: 9ec0  910003fd a90153f3 b00192b3 91138274 b9461680 340000e0 97fff8f4 340000c0
2024-09-26T11:32:36.067784+00:00 ubuntu kernel: 9ee0  a94153f3 a8c27bfd d50323bf d65f03c0 97fff621 52800020 b904e260 f942e293
2024-09-26T11:32:36.067909+00:00 ubuntu kernel: 9f00  b900069f b50000d3 97fffc3b b9461680 35fffe80 97fffd78 17fffff2 79409260
2024-09-26T11:32:36.067921+00:00 ubuntu kernel: 9f20  36100080 f9401660 b4000040 d63f0000 f9403a73 17fffff4 aa1e03e9 d503201f
2024-09-26T11:32:36.067922+00:00 ubuntu kernel: 9f40  d503233f a9be7bfd 910003fd a90153f3 b00192b3 91138273 2a0003f4 97fff8ea
2024-09-26T11:32:36.067923+00:00 ubuntu kernel: 9f60  b900067f 7100069f 540000c0 97fffc22 a94153f3 a8c27bfd d50323bf d65f03c0
2024-09-26T11:32:36.067923+00:00 ubuntu kernel: 9f80  90017560 f9467c00 9400066e f942e261 b4fffee1 f9002c20 f9403821 17fffffd
2024-09-26T11:32:36.067924+00:00 ubuntu kernel: 9fa0  aa1e03e9 d503201f d503233f a9bd7bfd 910003fd a90153f3 f90013f5 aa0003f5
2024-09-26T11:32:36.067924+00:00 ubuntu kernel: 9fc0  97fff5ed b00192a0 f9455014 b5000074 d2800013 1400000a f9401282 b5000062
2024-09-26T11:32:36.067925+00:00 ubuntu kernel: 9fe0  f9403a94 17fffffa aa1503e1 aa1403e0 d63f0040 aa0003f3 b4ffff40 97fffbfe
2024-09-26T11:32:36.067926+00:00 ubuntu kernel: a000  f94013f5 aa1303e0 a94153f3 a8c37bfd d50323bf d65f03c0 aa1e03e9 d503201f
2024-09-26T11:32:36.067927+00:00 ubuntu kernel: a020  d000ee60 39734000 36000120 d503233f a9bf7bfd 52800020 910003fd 97fff5e7
2024-09-26T11:32:36.067927+00:00 ubuntu kernel: a040* a8c17bfd d50323bf d65f03c0 d65f03c0 aa1e03e9 d503201f d000ee60 39734000
2024-09-26T11:32:36.067928+00:00 ubuntu kernel: a060  36000120 d503233f a9bf7bfd 52800060 910003fd 97fff5d9 a8c17bfd d50323bf
2024-09-26T11:32:36.067929+00:00 ubuntu kernel: a080  d65f03c0 d65f03c0 aa1e03e9 d503201f d503233f a9b87bfd 910003fd a9025bf5
2024-09-26T11:32:36.067930+00:00 ubuntu kernel: a0a0  d0017456 912be2d6 a90153f3 a90363f7 a9046bf9 f9002bfb b94012d3 35000c73
2024-09-26T11:32:36.067931+00:00 ubuntu kernel: a0c0  2a0003f7 b9400ec0 2a0103f4 aa0203f8 aa0303f9 aa0403fa 34000120 f00174e0
2024-09-26T11:32:36.067931+00:00 ubuntu kernel: a0e0  900141b5 910022b5 b94ae01b 97fff232 b8756800 6b1b001f 54000321 31000a9f
2024-09-26T11:32:36.067932+00:00 ubuntu kernel: a100  54000400 52800015 b94016d3 340000f3 d2912b16 f2a00836 aa1603e0 9443ed0f
2024-09-26T11:32:36.067932+00:00 ubuntu kernel: a120  71000673 54ffffa1 a9400740 a90607e0 910183e4 a9410740 aa1903e3 aa1803e2
2024-09-26T11:32:36.067933+00:00 ubuntu kernel: a140  a90707e0 2a1403e1 2a1703e0 97fffe1f 2a0003f3 340001d5 97ffffbe a9425bf5
2024-09-26T11:32:36.067934+00:00 ubuntu kernel: a160  2a1303e0 a94153f3 a94363f7 a9446bf9 f9402bfb a8c87bfd d50323bf d65f03c0
2024-09-26T11:32:36.067934+00:00 ubuntu kernel: a180  52800035 12800014 17ffffe0 97fff85e 35000580 f00174e0 b94ae000 3100041f
2024-09-26T11:32:36.067935+00:00 ubuntu kernel: a1a0  540003c1 900192b4 91138294 97fff284 f0017557 aa0003f6 913222f7 94000111
2024-09-26T11:32:36.067935+00:00 ubuntu kernel: a1c0  910502e0 9445e56d f942da80 3956e281 f100001f 12001c21 1a9f07e2 52000021
2024-09-26T11:32:36.067936+00:00 ubuntu kernel: a1e0  0a010058 6a01005f 540000c0 d5384101 eb01001f 54000160 52800020 3916e280
2024-09-26T11:32:36.067936+00:00 ubuntu kernel: a200  910502e0 9445e590 37000138 94000110 aa1603e0 97fff277 97ffff80 17ffffd0
2024-09-26T11:32:36.067937+00:00 ubuntu kernel: a220  2a1503f8 17fffff7 d503203f 3956e280 3707ffc0 94000106 aa1603e0 97fff26d
2024-09-26T11:32:36.067937+00:00 ubuntu kernel: 
2024-09-26T11:32:36.067938+00:00 ubuntu kernel: X19: 0xffff000100245000:
2024-09-26T11:32:36.067939+00:00 ubuntu kernel: 4e00  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067940+00:00 ubuntu kernel: 4e20  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067941+00:00 ubuntu kernel: 4e40  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067941+00:00 ubuntu kernel: 4e60  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067942+00:00 ubuntu kernel: 4e80  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067942+00:00 ubuntu kernel: 4ea0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067943+00:00 ubuntu kernel: 4ec0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067943+00:00 ubuntu kernel: 4ee0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067944+00:00 ubuntu kernel: 4f00  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067945+00:00 ubuntu kernel: 4f20  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067945+00:00 ubuntu kernel: 4f40  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067946+00:00 ubuntu kernel: 4f60  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067946+00:00 ubuntu kernel: 4f80  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067947+00:00 ubuntu kernel: 4fa0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067948+00:00 ubuntu kernel: 4fc0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067948+00:00 ubuntu kernel: 4fe0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.067949+00:00 ubuntu kernel: 5000* 013c4810 ffff0001 0389bd00 ffff0001 043edc18 ffff0001 0097dc08 ffff0001
2024-09-26T11:32:36.067950+00:00 ubuntu kernel: 5020  038a0580 ffff0001 00cdc000 ffff0001 0b1b4198 ffff8000 0389fc80 ffff0001
2024-09-26T11:32:36.068024+00:00 ubuntu kernel: 5040  00000005 00000007 013c4810 ffff0001 0030c600 ffff0001 00000000 00000000
2024-09-26T11:32:36.068025+00:00 ubuntu kernel: 5060  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068026+00:00 ubuntu kernel: 5080  00000000 00000000 00000000 00000000 00000000 dead4ead ffffffff 00000000
2024-09-26T11:32:36.068027+00:00 ubuntu kernel: 50a0  ffffffff ffffffff 00000000 00000000 002450b0 ffff0001 002450b0 ffff0001
2024-09-26T11:32:36.068027+00:00 ubuntu kernel: 50c0  002450c0 ffff0001 002450c0 ffff0001 002450d0 ffff0001 002450d0 ffff0001
2024-09-26T11:32:36.068028+00:00 ubuntu kernel: 50e0  002450e0 ffff0001 002450e0 ffff0001 00000000 00000000 00000000 00000106
2024-09-26T11:32:36.068029+00:00 ubuntu kernel: 5100  00000000 00000000 00000000 dead4ead ffffffff 00000000 ffffffff ffffffff
2024-09-26T11:32:36.068029+00:00 ubuntu kernel: 5120  043edd28 ffff0001 0097dd18 ffff0001 ffffffff 00000000 00000000 dead4ead
2024-09-26T11:32:36.068030+00:00 ubuntu kernel: 5140  ffffffff 00000000 ffffffff ffffffff 00245150 ffff0001 00245150 ffff0001
2024-09-26T11:32:36.068030+00:00 ubuntu kernel: 5160  00000000 00000000 00000000 00000000 00245170 ffff0001 00000000 00000000
2024-09-26T11:32:36.068031+00:00 ubuntu kernel: 5180  00000000 00000000 00000000 00000000 00000000 00000000 08995664 ffff8000
2024-09-26T11:32:36.068032+00:00 ubuntu kernel: 51a0  fddac480 ffff0004 00000000 00000000 00000000 00000000 ffffffe0 0000000f
2024-09-26T11:32:36.068032+00:00 ubuntu kernel: 51c0  002451c0 ffff0001 002451c0 ffff0001 08995c9c ffff8000 00000000 dead4ead
2024-09-26T11:32:36.068033+00:00 ubuntu kernel: 51e0  ffffffff 00000000 ffffffff ffffffff 002451f0 ffff0001 002451f0 ffff0001
2024-09-26T11:32:36.068033+00:00 ubuntu kernel: 
2024-09-26T11:32:36.068034+00:00 ubuntu kernel: X20: 0xffff0001003a9680:
2024-09-26T11:32:36.068035+00:00 ubuntu kernel: 9480  00000104 00004000 00000000 00000000 0001916a 00000000 00000000 00000000
2024-09-26T11:32:36.068035+00:00 ubuntu kernel: 94a0  00000000 0000000b 8000000a 00000000 00000000 dead4ead ffffffff 00000000
2024-09-26T11:32:36.068036+00:00 ubuntu kernel: 94c0  ffffffff ffffffff 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068036+00:00 ubuntu kernel: 94e0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 dead4ead
2024-09-26T11:32:36.068037+00:00 ubuntu kernel: 9500  ffffffff 00000000 ffffffff ffffffff 003a9510 ffff0001 003a9510 ffff0001
2024-09-26T11:32:36.068038+00:00 ubuntu kernel: 9520  00000001 00000000 00000000 00000000 002f3c00 ffff0001 00000000 00000000
2024-09-26T11:32:36.068039+00:00 ubuntu kernel: 9540  00000000 00000000 0389bf00 ffff0001 003a9550 ffff0001 003a9550 ffff0001
2024-09-26T11:32:36.068039+00:00 ubuntu kernel: 9560  01182f00 ffff0001 00000000 00000000 0b076160 ffff8000 0389e780 ffff0001
2024-09-26T11:32:36.068040+00:00 ubuntu kernel: 9580  00000001 00000003 00000000 00000000 00000000 dead4ead ffffffff 00000000
2024-09-26T11:32:36.068041+00:00 ubuntu kernel: 95a0  ffffffff ffffffff 00000000 00000000 003a95b0 ffff0001 003a95b0 ffff0001
2024-09-26T11:32:36.068041+00:00 ubuntu kernel: 95c0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068042+00:00 ubuntu kernel: 95e0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068042+00:00 ubuntu kernel: 9600  0097d400 ffff0001 0030c200 ffff0001 0898a830 ffff8000 09dce3f9 ffff8000
2024-09-26T11:32:36.068043+00:00 ubuntu kernel: 9620  000000f8 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068043+00:00 ubuntu kernel: 9640  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068044+00:00 ubuntu kernel: 9660  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068045+00:00 ubuntu kernel: 9680* 09891eb8 ffff8000 0cc30180 ffff8000 00245000 ffff0001 00000001 dead4ead
2024-09-26T11:32:36.068046+00:00 ubuntu kernel: 96a0  00000000 00000000 04cacec0 ffff0001 00000001 00000000 00000020 00000000
2024-09-26T11:32:36.068046+00:00 ubuntu kernel: 96c0  0000001f 00000000 00000000 01000001 00000000 00000000 00000200 00000000
2024-09-26T11:32:36.068047+00:00 ubuntu kernel: 96e0  000f8000 00000000 0c405000 ffff8000 00080810 00000000 0ce82000 ffff8000
2024-09-26T11:32:36.068047+00:00 ubuntu kernel: 9700  00000008 00000000 0383e000 00000000 10700000 00000000 00000000 00000000
2024-09-26T11:32:36.068048+00:00 ubuntu kernel: 9720  00000000 dead4ead ffffffff 00000000 ffffffff ffffffff 003a9738 ffff0001
2024-09-26T11:32:36.068048+00:00 ubuntu kernel: 9740  003a9738 ffff0001 00000000 dead4ead ffffffff 00000000 ffffffff ffffffff
2024-09-26T11:32:36.068049+00:00 ubuntu kernel: 9760  003a9760 ffff0001 003a9760 ffff0001 0097d480 ffff0001 00000000 00000000
2024-09-26T11:32:36.068050+00:00 ubuntu kernel: 9780  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068050+00:00 ubuntu kernel: 97a0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068051+00:00 ubuntu kernel: 97c0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068051+00:00 ubuntu kernel: 97e0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068052+00:00 ubuntu kernel: 9800  fc590678 ffff0003 fc590678 ffff0003 003a9a10 ffff0001 0444d410 ffff0001
2024-09-26T11:32:36.068053+00:00 ubuntu kernel: 9820  00000000 00000000 00000000 dead4ead ffffffff 00000000 ffffffff ffffffff
2024-09-26T11:32:36.068054+00:00 ubuntu kernel: 9840  00000000 00000000 003a9848 ffff0001 003a9848 ffff0001 00000000 00000000
2024-09-26T11:32:36.068055+00:00 ubuntu kernel: 9860  00000000 00000000 00000000 00000000 003a9870 ffff0001 003a9870 ffff0001
2024-09-26T11:32:36.068055+00:00 ubuntu kernel: 
2024-09-26T11:32:36.068056+00:00 ubuntu kernel: X23: 0xffff80000b2e6160:
2024-09-26T11:32:36.068057+00:00 ubuntu kernel: 5f60  09890154 ffff8000 09e5fb1e ffff8000 09e5f98a ffff8000 00fc02fc 00000000
2024-09-26T11:32:36.068057+00:00 ubuntu kernel: 5f80  09e5fb17 ffff8000 0989016a ffff8000 09e5fb1e ffff8000 09e5f98a ffff8000
2024-09-26T11:32:36.068058+00:00 ubuntu kernel: 5fa0  00fc02ab 00000000 09e5fb17 ffff8000 098900fd ffff8000 09e5fb1e ffff8000
2024-09-26T11:32:36.068058+00:00 ubuntu kernel: 5fc0  09e5fa2a ffff8000 00fc049a 00000000 09e5fb17 ffff8000 098901a5 ffff8000
2024-09-26T11:32:36.068059+00:00 ubuntu kernel: 5fe0  09e5fb1e ffff8000 09e5fab7 ffff8000 00fc054f 00000000 09e5fb17 ffff8000
2024-09-26T11:32:36.068060+00:00 ubuntu kernel: 6000  09890180 ffff8000 09e5fb1e ffff8000 09e5fafc ffff8000 00fc0810 00000000
2024-09-26T11:32:36.068060+00:00 ubuntu kernel: 6020  09e5fda7 ffff8000 09890828 ffff8000 09e5fdb7 ffff8000 09e5fc38 ffff8000
2024-09-26T11:32:36.068061+00:00 ubuntu kernel: 6040  00fc0209 00000000 09e5fda7 ffff8000 09890828 ffff8000 09e5fdb7 ffff8000
2024-09-26T11:32:36.068062+00:00 ubuntu kernel: 6060  09e5fc5c ffff8000 00fc020d 00000000 09e5fda7 ffff8000 098905b8 ffff8000
2024-09-26T11:32:36.068062+00:00 ubuntu kernel: 6080  09e5fdb7 ffff8000 09e5fc76 ffff8000 00fc0094 00000000 09e5fda7 ffff8000
2024-09-26T11:32:36.068063+00:00 ubuntu kernel: 60a0  0989083f ffff8000 09e5fdb7 ffff8000 09e5fd4d ffff8000 00fc01b4 00000000
2024-09-26T11:32:36.068064+00:00 ubuntu kernel: 60c0  09e5fda7 ffff8000 0989083f ffff8000 09e5fdb7 ffff8000 09e5fd6e ffff8000
2024-09-26T11:32:36.068064+00:00 ubuntu kernel: 60e0  00fc01bd 00000000 09e602c8 ffff8000 09890c40 ffff8000 09e602d8 ffff8000
2024-09-26T11:32:36.068065+00:00 ubuntu kernel: 6100  09e5ffa0 ffff8000 00fc0418 00000000 09e604b8 ffff8000 09891f48 ffff8000
2024-09-26T11:32:36.068066+00:00 ubuntu kernel: 6120  09e604c9 ffff8000 09e60355 ffff8000 00fc028a 00000000 09e604b8 ffff8000
2024-09-26T11:32:36.068067+00:00 ubuntu kernel: 6140  09891f48 ffff8000 09e604c9 ffff8000 09e6038e ffff8000 00fc0293 00000000
2024-09-26T11:32:36.068067+00:00 ubuntu kernel: 6160* 09e6067b ffff8000 098928c0 ffff8000 09e60cc9 ffff8000 09e604ed ffff8000
2024-09-26T11:32:36.068068+00:00 ubuntu kernel: 6180  00fc02a2 00000000 09e6067b ffff8000 098928d6 ffff8000 09e60cc9 ffff8000
2024-09-26T11:32:36.068069+00:00 ubuntu kernel: 61a0  09e60509 ffff8000 00fc02c4 00000000 09e6067b ffff8000 098928e6 ffff8000
2024-09-26T11:32:36.068069+00:00 ubuntu kernel: 61c0  09e60cc9 ffff8000 09e60600 ffff8000 00fc033b 00000000 09e6067b ffff8000
2024-09-26T11:32:36.068070+00:00 ubuntu kernel: 61e0  098928e6 ffff8000 09e60cc9 ffff8000 09e6061f ffff8000 00fc034c 00000000
2024-09-26T11:32:36.068071+00:00 ubuntu kernel: 6200  09e6067b ffff8000 098928e6 ffff8000 09e60cc9 ffff8000 09e60638 ffff8000
2024-09-26T11:32:36.068072+00:00 ubuntu kernel: 6220  00fc0353 00000000 09e6067b ffff8000 098928af ffff8000 09e60cc9 ffff8000
2024-09-26T11:32:36.068072+00:00 ubuntu kernel: 6240  09e60681 ffff8000 00fc047b 00000000 09e6067b ffff8000 098928af ffff8000
2024-09-26T11:32:36.068073+00:00 ubuntu kernel: 6260  09e60cc9 ffff8000 09e606b1 ffff8000 00fc0489 00000000 09e6067b ffff8000
2024-09-26T11:32:36.068074+00:00 ubuntu kernel: 6280  098927c0 ffff8000 09e60cc9 ffff8000 09e609b4 ffff8000 00fc03e9 00000000
2024-09-26T11:32:36.068074+00:00 ubuntu kernel: 62a0  09e6067b ffff8000 098927d5 ffff8000 09e60cc9 ffff8000 09e609b4 ffff8000
2024-09-26T11:32:36.068075+00:00 ubuntu kernel: 62c0  00fc0435 00000000 09e6067b ffff8000 098927d5 ffff8000 09e60cc9 ffff8000
2024-09-26T11:32:36.068080+00:00 ubuntu kernel: 62e0  09e609e7 ffff8000 00fc0463 00000000 09e6067b ffff8000 09892897 ffff8000
2024-09-26T11:32:36.068080+00:00 ubuntu kernel: 6300  09e60cc9 ffff8000 09e60a53 ffff8000 00fc026b 00000000 09e6067b ffff8000
2024-09-26T11:32:36.068081+00:00 ubuntu kernel: 6320  09892882 ffff8000 09e60cc9 ffff8000 09e60a92 ffff8000 00fc01c1 00000000
2024-09-26T11:32:36.068082+00:00 ubuntu kernel: 6340  09e6067b ffff8000 09892882 ffff8000 09e60cc9 ffff8000 09e60ac6 ffff8000
2024-09-26T11:32:36.068082+00:00 ubuntu kernel: 
2024-09-26T11:32:36.068083+00:00 ubuntu kernel: X24: 0xffff80000b298eef:
2024-09-26T11:32:36.068083+00:00 ubuntu kernel: 8cec  00000000 fc592268 ffff0003 fc592150 ffff0003 08195dfc ffff8000 099700ab
2024-09-26T11:32:36.068084+00:00 ubuntu kernel: 8d0c  ffff8000 091f1e28 ffff8000 091f1810 ffff8000 08193710 ffff8000 0b282bd0
2024-09-26T11:32:36.068086+00:00 ubuntu kernel: 8d2c  ffff8000 00000000 00000000 fc592118 ffff0003 fc592000 ffff0003 08195dfc
2024-09-26T11:32:36.068086+00:00 ubuntu kernel: 8d4c  ffff8000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068138+00:00 ubuntu kernel: 8d6c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068138+00:00 ubuntu kernel: 8d8c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068139+00:00 ubuntu kernel: 8dac  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068140+00:00 ubuntu kernel: 8dcc  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068141+00:00 ubuntu kernel: 8dec  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068141+00:00 ubuntu kernel: 8e0c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068143+00:00 ubuntu kernel: 8e2c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00010000
2024-09-26T11:32:36.068143+00:00 ubuntu kernel: 8e4c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068144+00:00 ubuntu kernel: 8e6c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068145+00:00 ubuntu kernel: 8e8c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068146+00:00 ubuntu kernel: 8eac  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068147+00:00 ubuntu kernel: 8ecc  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068147+00:00 ubuntu kernel: 8eec* 00000000 00000100 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068148+00:00 ubuntu kernel: 8f0c  01010000 00000001 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068148+00:00 ubuntu kernel: 8f2c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068149+00:00 ubuntu kernel: 8f4c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068154+00:00 ubuntu kernel: 8f6c  00000000 00000000 00000000 00000000 00000000 01000000 00000001 00010000
2024-09-26T11:32:36.068155+00:00 ubuntu kernel: 8f8c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068157+00:00 ubuntu kernel: 8fac  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068158+00:00 ubuntu kernel: 8fcc  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068159+00:00 ubuntu kernel: 8fec  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068160+00:00 ubuntu kernel: 900c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068162+00:00 ubuntu kernel: 902c  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068163+00:00 ubuntu kernel: 904c  00000000 00000000 00000000 00000000 00000000 09ebf3d3 ffff8000 00000000
2024-09-26T11:32:36.068164+00:00 ubuntu kernel: 906c  00000000 0af7be58 ffff8000 00000000 00000000 080144dc ffff8000 00000000
2024-09-26T11:32:36.068166+00:00 ubuntu kernel: 908c  00000000 00000000 00000000 00000000 00000000 09ebf3c4 ffff8000 00000000
2024-09-26T11:32:36.068167+00:00 ubuntu kernel: 90ac  00000000 0af7be50 ffff8000 00000000 00000000 08014534 ffff8000 00000000
2024-09-26T11:32:36.068169+00:00 ubuntu kernel: 90cc  00000000 00000000 00000000 00000000 00000000 09ebf3b4 ffff8000 00000000
2024-09-26T11:32:36.068170+00:00 ubuntu kernel: 90ec  00000000 0af7be48 ffff8000 00000000 00000000 0801458c ffff8000 00000000
2024-09-26T11:32:36.068172+00:00 ubuntu kernel: 
2024-09-26T11:32:36.068174+00:00 ubuntu kernel: X25: 0xffff0001003a9698:
2024-09-26T11:32:36.068180+00:00 ubuntu kernel: 9498  00000000 00000000 00000000 0000000b 8000000a 00000000 00000000 dead4ead
2024-09-26T11:32:36.068182+00:00 ubuntu kernel: 94b8  ffffffff 00000000 ffffffff ffffffff 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068184+00:00 ubuntu kernel: 94d8  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068185+00:00 ubuntu kernel: 94f8  00000000 dead4ead ffffffff 00000000 ffffffff ffffffff 003a9510 ffff0001
2024-09-26T11:32:36.068187+00:00 ubuntu kernel: 9518  003a9510 ffff0001 00000001 00000000 00000000 00000000 002f3c00 ffff0001
2024-09-26T11:32:36.068188+00:00 ubuntu kernel: 9538  00000000 00000000 00000000 00000000 0389bf00 ffff0001 003a9550 ffff0001
2024-09-26T11:32:36.068190+00:00 ubuntu kernel: 9558  003a9550 ffff0001 01182f00 ffff0001 00000000 00000000 0b076160 ffff8000
2024-09-26T11:32:36.068191+00:00 ubuntu kernel: 9578  0389e780 ffff0001 00000001 00000003 00000000 00000000 00000000 dead4ead
2024-09-26T11:32:36.068193+00:00 ubuntu kernel: 9598  ffffffff 00000000 ffffffff ffffffff 00000000 00000000 003a95b0 ffff0001
2024-09-26T11:32:36.068194+00:00 ubuntu kernel: 95b8  003a95b0 ffff0001 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068196+00:00 ubuntu kernel: 95d8  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068325+00:00 ubuntu kernel: 95f8  00000000 00000000 0097d400 ffff0001 0030c200 ffff0001 0898a830 ffff8000
2024-09-26T11:32:36.068331+00:00 ubuntu kernel: 9618  09dce3f9 ffff8000 000000f8 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068332+00:00 ubuntu kernel: 9638  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068333+00:00 ubuntu kernel: 9658  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068334+00:00 ubuntu kernel: 9678  00000000 00000000 09891eb8 ffff8000 0cc30180 ffff8000 00245000 ffff0001
2024-09-26T11:32:36.068335+00:00 ubuntu kernel: 9698* 00000001 dead4ead 00000000 00000000 04cacec0 ffff0001 00000001 00000000
2024-09-26T11:32:36.068335+00:00 ubuntu kernel: 96b8  00000020 00000000 0000001f 00000000 00000000 01000001 00000000 00000000
2024-09-26T11:32:36.068336+00:00 ubuntu kernel: 96d8  00000200 00000000 000f8000 00000000 0c405000 ffff8000 00080810 00000000
2024-09-26T11:32:36.068337+00:00 ubuntu kernel: 96f8  0ce82000 ffff8000 00000008 00000000 0383e000 00000000 10700000 00000000
2024-09-26T11:32:36.068337+00:00 ubuntu kernel: 9718  00000000 00000000 00000000 dead4ead ffffffff 00000000 ffffffff ffffffff
2024-09-26T11:32:36.068338+00:00 ubuntu kernel: 9738  003a9738 ffff0001 003a9738 ffff0001 00000000 dead4ead ffffffff 00000000
2024-09-26T11:32:36.068339+00:00 ubuntu kernel: 9758  ffffffff ffffffff 003a9760 ffff0001 003a9760 ffff0001 0097d480 ffff0001
2024-09-26T11:32:36.068340+00:00 ubuntu kernel: 9778  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068341+00:00 ubuntu kernel: 9798  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068341+00:00 ubuntu kernel: 97b8  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068342+00:00 ubuntu kernel: 97d8  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068343+00:00 ubuntu kernel: 97f8  00000000 00000000 fc590678 ffff0003 fc590678 ffff0003 003a9a10 ffff0001
2024-09-26T11:32:36.068344+00:00 ubuntu kernel: 9818  0444d410 ffff0001 00000000 00000000 00000000 dead4ead ffffffff 00000000
2024-09-26T11:32:36.068344+00:00 ubuntu kernel: 9838  ffffffff ffffffff 00000000 00000000 003a9848 ffff0001 003a9848 ffff0001
2024-09-26T11:32:36.068345+00:00 ubuntu kernel: 9858  00000000 00000000 00000000 00000000 00000000 00000000 003a9870 ffff0001
2024-09-26T11:32:36.068350+00:00 ubuntu kernel: 9878  003a9870 ffff0001 003a9880 ffff0001 003a9880 ffff0001 003a9890 ffff0001
2024-09-26T11:32:36.068350+00:00 ubuntu kernel: 
2024-09-26T11:32:36.068351+00:00 ubuntu kernel: X29: 0xffff800008003dc0:
2024-09-26T11:32:36.068352+00:00 ubuntu kernel: 3bc0  08003c60 ffff8000 ffffffff 00000000 08003c20 ffff8000 0925c860 ffff8000
2024-09-26T11:32:36.068352+00:00 ubuntu kernel: 3be0  08003c60 ffff8000 0a91e2f4 ffff8000 f2000800 00000000 44000080 f7e1fddf
2024-09-26T11:32:36.068353+00:00 ubuntu kernel: 3c00  604000c9 00000000 0b298eef ffff8000 003a9698 ffff0001 010f0001 00000000
2024-09-26T11:32:36.068354+00:00 ubuntu kernel: 3c20  08003c50 ffff8000 0925cc80 ffff8000 00245000 ffff0001 003a9680 ffff0001
2024-09-26T11:32:36.068354+00:00 ubuntu kernel: 3c40  08003db0 ffff8000 08d8d640 ffff8000 08003d90 ffff8000 080112e0 ffff8000
2024-09-26T11:32:36.068355+00:00 ubuntu kernel: 3c60  04cacec0 ffff0001 00000000 00000000 00000000 00000000 00000027 00000000
2024-09-26T11:32:36.068356+00:00 ubuntu kernel: 3c80  00000000 00000000 fdda9a90 ffff0004 30636d6d 7163203a 3a696368 75707320
2024-09-26T11:32:36.068357+00:00 ubuntu kernel: 3ca0  756f6972 43542073 080ea040 ffff8000 63203a30 69636871 7073203a 6f697275
2024-09-26T11:32:36.068358+00:00 ubuntu kernel: 3cc0  54207375 66204e43 7420726f 39206761 00000000 00000000 00000000 00000000
2024-09-26T11:32:36.068358+00:00 ubuntu kernel: 3ce0  00000000 00000000 00000000 00000000 00000000 00000000 00245000 ffff0001
2024-09-26T11:32:36.068359+00:00 ubuntu kernel: 3d00  003a9680 ffff0001 00000002 00000000 00000009 00000000 0b2e6160 ffff8000
2024-09-26T11:32:36.068360+00:00 ubuntu kernel: 3d20  0b298eef ffff8000 003a9698 ffff0001 010f0001 00000000 0270003e 00000000
2024-09-26T11:32:36.068360+00:00 ubuntu kernel: 3d40  03ff003f 00000000 08003dc0 ffff8000 08d8d640 ffff8000 08003db0 ffff8000
2024-09-26T11:32:36.068361+00:00 ubuntu kernel: 3d60  08d8d640 ffff8000 604000c9 00000000 08003da0 ffff8000 0389bd00 ffff0001
2024-09-26T11:32:36.068362+00:00 ubuntu kernel: 3d80  00000009 00000000 0097d480 ffff0001 08003dc0 ffff8000 08d8d640 ffff8000
2024-09-26T11:32:36.068363+00:00 ubuntu kernel: 3da0  00000000 00000000 00000000 00000000 00000200 00000000 943f8800 498c6f08
2024-09-26T11:32:36.068365+00:00 ubuntu kernel: 3dc0* 08003e20 ffff8000 08d8bad0 ffff8000 00000000 00000000 00245600 ffff0001
2024-09-26T11:32:36.068365+00:00 ubuntu kernel: 3de0  003a9400 ffff0001 00000000 00000000 0b2e5af8 ffff8000 002458f0 ffff0001
2024-09-26T11:32:36.068366+00:00 ubuntu kernel: 3e00  00000010 00000000 08d8bab8 ffff8000 00000000 00000000 943f8800 498c6f08
2024-09-26T11:32:36.068367+00:00 ubuntu kernel: 3e20  08003e60 ffff8000 08d839a4 ffff8000 00245600 ffff0001 00000000 00000000
2024-09-26T11:32:36.068367+00:00 ubuntu kernel: 3e40  fdda9f00 00004000 00000000 00000000 00000000 00000000 943f8800 498c6f08
2024-09-26T11:32:36.068368+00:00 ubuntu kernel: 3e60  08003ec0 ffff8000 080ed084 ffff8000 038a0280 ffff0001 00000061 00000000
2024-09-26T11:32:36.068397+00:00 ubuntu kernel: 3e80  003a9400 ffff0001 003a94b0 ffff0001 00000000 00000000 0b299da0 ffff8000
2024-09-26T11:32:36.068398+00:00 ubuntu kernel: 3ea0  0b299de0 ffff8000 00000000 00000000 00000001 00000000 04cacec0 ffff0001
2024-09-26T11:32:36.068399+00:00 ubuntu kernel: 3ec0  08003f10 ffff8000 080ed1a8 ffff8000 003a9400 ffff0001 0b135b10 ffff8000
2024-09-26T11:32:36.068400+00:00 ubuntu kernel: 3ee0  003a94b0 ffff0001 003a94b0 ffff0001 60400009 00000000 0000000a 00000000
2024-09-26T11:32:36.068401+00:00 ubuntu kernel: 3f00  00000001 00000000 00000000 00000000 08003f30 ffff8000 080ed22c ffff8000
2024-09-26T11:32:36.068402+00:00 ubuntu kernel: 3f20  003a9400 ffff0001 0b135b10 ffff8000 08003f60 ffff8000 080f27d4 ffff8000
2024-09-26T11:32:36.068404+00:00 ubuntu kernel: 3f40  003a9400 ffff0001 0b135b10 ffff8000 00004000 00000000 080f2764 ffff8000
2024-09-26T11:32:36.068406+00:00 ubuntu kernel: 3f60  08003fa0 ffff8000 080ec7f4 ffff8000 000000ed 00000000 00000000 00000000
2024-09-26T11:32:36.068407+00:00 ubuntu kernel: 3f80  00000000 00000000 08d7e470 ffff8000 60400009 00000000 08d5bf38 ffff8000
2024-09-26T11:32:36.068409+00:00 ubuntu kernel: 3fa0  08003fb0 ffff8000 080ec8a4 ffff8000 08003fc0 ffff8000 086ac494 ffff8000
2024-09-26T11:32:36.068411+00:00 ubuntu kernel: ---[ end trace 0000000000000000 ]---
2024-09-26T11:32:36.071757+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:36.588389+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:36.588403+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:37.095098+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:40.281757+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:40.801762+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:40.805057+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:41.335075+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:41.335107+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:41.841740+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:41.841770+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:42.348472+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:42.365103+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:42.881790+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:50.268454+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:50.775083+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:50.775119+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:51.281771+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:51.281790+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:51.788383+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:51.788399+00:00 ubuntu kernel: mmc0: running CQE recovery
2024-09-26T11:32:52.295097+00:00 ubuntu kernel: mmc0: cqhci: Failed to halt
2024-09-26T11:32:52.295129+00:00 ubuntu kernel: I/O error, dev mmcblk0, sector 4047560 op 0x1:(WRITE) flags 0x9800 phys_seg 21 prio class 2
2024-09-26T11:32:52.295132+00:00 ubuntu kernel: Aborting journal on device mmcblk0p2-8.
2024-09-26T11:32:52.298408+00:00 ubuntu kernel: EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 257144, error -30)
2024-09-26T11:32:52.298426+00:00 ubuntu kernel: EXT4-fs error (device mmcblk0p2): ext4_journal_check_start:83: comm rs:main Q:Reg: Detected aborted journal
2024-09-26T11:32:52.298428+00:00 ubuntu kernel: EXT4-fs error (device mmcblk0p2): ext4_journal_check_start:83: comm journal-offline: Detected aborted journal
2024-09-26T11:32:52.301727+00:00 ubuntu kernel: EXT4-fs error (device mmcblk0p2): ext4_journal_check_start:83: comm systemd-journal: Detected aborted journal
2024-09-26T11:32:52.301738+00:00 ubuntu kernel: EXT4-fs error (device mmcblk0p2): ext4_journal_check_start:83: comm .NET TP Worker: Detected aborted journal
2024-09-26T11:32:52.301738+00:00 ubuntu kernel: EXT4-fs (mmcblk0p2): Remounting filesystem read-only
2024-09-26T11:32:52.301740+00:00 ubuntu kernel: EXT4-fs (mmcblk0p2): Remounting filesystem read-only
2024-09-26T11:32:52.301746+00:00 ubuntu kernel: EXT4-fs (mmcblk0p2): ext4_writepages: jbd2_start: 9223372036854775769 pages, ino 2037; err -30
2024-09-26T11:32:52.304982+00:00 ubuntu kernel: audit: type=1400 audit(1727350372.300:52): apparmor="DENIED" operation="open" profile="rsyslogd" name="/run/systemd/sessions/" pid=881 comm=72733A6D61696E20513A526567 requested_mask="r" denied_mask="r" fsuid=102 ouid=0
tomharvey commented 2 months ago

After a reboot I ended up in busy box.

After running fsck -y on the device I can exit out of busy box and all is well again... I assume, until the next time the queue gets borked

HeyMeco commented 1 month ago

After a reboot I ended up in busy box.

After running fsck -y on the device I can exit out of busy box and all is well again... I assume, until the next time the queue gets borked

Had the same thing happen. The above fix also worked on mine

Joshua-Riek commented 1 month ago

Interesting, dropping the eMMC down to HS200 is the proposed change here?

While this is not a fix, dropping https://github.com/Joshua-Riek/linux-rockchip/commit/d03a5046f13a8a5b4d7c4b02e442aa4508890413 may be a better option, but how will this affect other devices?

HeyMeco commented 1 month ago

@Joshua-Riek

how will this affect other devices?

Should only reduce IOPS a bit but better to have a stable device

nyanmisaka commented 1 month ago

As an alternative, add these only to known affected devices?

&sdhci {
...
    /delete-property/ supports-cqe;
...
};
HeyMeco commented 4 weeks ago

I think I need to try a dtbo like this on Rock5B:

/dts-v1/;
/plugin/;

/ {
    fragment@0 {
        target = <&sdhci>;
        __overlay__ {
            mmc@fe2e0000 {
                /* Commenting out supports-cqe */
                // supports-cqe;
            };
        };
    };
};