search

openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.42k stars 1.72k forks source link

some zvols froze at random (could be related to a snapshot) #11143

Closed Pentium100MHz closed 2 years ago

Pentium100MHz commented 3 years ago

System information

Type Version/Name
Distribution Name Debian
Distribution Version 10.5
Linux Kernel 4.19.0-10-amd64
Architecture amd64
ZFS Version 0.8.4-2~bpo10+1
SPL Version 0.8.4-2~bpo10+1

Describe the problem you're observing

Some zfs related processes stuck in D state, almost zero disk IO. Some VMs that use zvols as storage had frozen (unable to access the storage), some seemed to work, but I did not check it thoroughly, just reset the whole server. There were no qemu processes stuck in D state, just the zfs related ones. The server worked fine for a couple of months before this happened. After reset it appears to work fine.

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root       573  0.0  0.0  11604  4388 ?        D    05:51   0:00 /sbin/zfs snapshot -r home/zfs-2988@r2020-10-31
root     35317  0.0  0.0      0     0 ?        D<   05:51   0:00 [z_wr_int]
root     35321  0.0  0.0      0     0 ?        D<   05:51   0:00 [z_wr_int]
root     35331  0.0  0.0      0     0 ?        D<   05:51   0:00 [z_wr_int]
root     35351  0.0  0.0      0     0 ?        D<   05:51   0:00 [z_wr_int]
root     35400  0.0  0.0      0     0 ?        D<   05:51   0:00 [z_wr_iss_h]
root     35432  0.0  0.0      0     0 ?        D<   05:51   0:00 [z_wr_iss_h]
root     35438  0.0  0.0      0     0 ?        D<   05:51   0:00 [z_wr_iss_h]
root     35448  0.0  0.0      0     0 ?        D<   05:51   0:00 [z_wr_iss_h]
root     38072  0.3  0.0      0     0 ?        D<   Sep22 199:53 [z_wr_iss]
root     38075  2.5  0.0      0     0 ?        D<   Sep22 1429:14 [z_wr_iss_h]
root     38076  0.4  0.0      0     0 ?        D<   Sep22 247:29 [z_wr_int]
root     38078  0.4  0.0      0     0 ?        D<   Sep22 247:24 [z_wr_int]
root     38079  0.4  0.0      0     0 ?        D<   Sep22 247:32 [z_wr_int]
root     38080  0.4  0.0      0     0 ?        D<   Sep22 247:28 [z_wr_int]
root     38081  0.4  0.0      0     0 ?        D<   Sep22 247:27 [z_wr_int]
root     38188  0.2  0.0      0     0 ?        D    Sep22 136:15 [txg_sync]

Describe how to reproduce the problem

This happened at random. There are a bunch of VMs running that use zvols as storage.

There are also snapshots used, the backup script does zfs snapshot newsnap, zfs send -i oldsnap newsnap; zfs destroy oldsnap and some zfs list commands.

Include any warning/errors/backtraces from the system logs

[3428822.298868] INFO: task z_wr_iss:38072 blocked for more than 120 seconds.
[3428822.300760]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.302496] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.304318] z_wr_iss        D    0 38072      2 0x80000000
[3428822.304326] Call Trace:
[3428822.304342]  __schedule+0x2a2/0x870
[3428822.304352]  schedule+0x28/0x80
[3428822.304360]  io_schedule+0x12/0x40
[3428822.304368]  wbt_wait+0x205/0x300
[3428822.304371]  ? wbt_wait+0x300/0x300
[3428822.304379]  rq_qos_throttle+0x31/0x40
[3428822.304386]  blk_mq_make_request+0x111/0x530
[3428822.304393]  generic_make_request+0x1a4/0x400
[3428822.304485]  ? abd_iter_advance+0x42/0x60 [zfs]
[3428822.304493]  submit_bio+0x45/0x130
[3428822.304576]  vdev_disk_io_start+0x6ee/0x9c0 [zfs]
[3428822.304648]  ? abd_copy_from_buf_off_cb+0x20/0x20 [zfs]
[3428822.304658]  ? ktime_get_raw_ts64+0x32/0xc0
[3428822.304753]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.304835]  zio_nowait+0xad/0x140 [zfs]
[3428822.304918]  vdev_queue_io+0x20d/0x220 [zfs]
[3428822.304999]  zio_vdev_io_start+0x9c/0x2e0 [zfs]
[3428822.305082]  ? vdev_raidz_asize+0x60/0x60 [zfs]
[3428822.305165]  zio_nowait+0xad/0x140 [zfs]
[3428822.305248]  vdev_raidz_io_start+0x218/0x2d0 [zfs]
[3428822.305330]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.305414]  ? vdev_config_sync+0x230/0x230 [zfs]
[3428822.305495]  zio_nowait+0xad/0x140 [zfs]
[3428822.305578]  vdev_mirror_io_start+0x97/0x180 [zfs]
[3428822.305681]  ? spa_config_enter+0xb2/0x100 [zfs]
[3428822.305773]  zio_vdev_io_start+0x259/0x2e0 [zfs]
[3428822.305785]  ? tsd_get_by_thread+0x2a/0x40 [spl]
[3428822.305793]  ? taskq_member+0x14/0x20 [spl]
[3428822.305872]  zio_execute+0x90/0xf0 [zfs]
[3428822.305884]  taskq_thread+0x2df/0x520 [spl]
[3428822.305890]  ? wake_up_q+0x70/0x70
[3428822.305970]  ? zio_taskq_member.isra.8.constprop.13+0x70/0x70 [zfs]
[3428822.305979]  ? taskq_thread_spawn+0x50/0x50 [spl]
[3428822.305986]  kthread+0x112/0x130
[3428822.305989]  ? kthread_bind+0x30/0x30
[3428822.305995]  ret_from_fork+0x35/0x40
[3428822.306000] INFO: task z_wr_iss_h:38075 blocked for more than 120 seconds.
[3428822.307808]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.309606] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.311423] z_wr_iss_h      D    0 38075      2 0x80000000
[3428822.311432] Call Trace:
[3428822.311438]  __schedule+0x2a2/0x870
[3428822.311442]  schedule+0x28/0x80
[3428822.311446]  io_schedule+0x12/0x40
[3428822.311449]  wbt_wait+0x205/0x300
[3428822.311452]  ? wbt_wait+0x300/0x300
[3428822.311458]  rq_qos_throttle+0x31/0x40
[3428822.311466]  blk_mq_make_request+0x111/0x530
[3428822.311473]  generic_make_request+0x1a4/0x400
[3428822.311479]  submit_bio+0x45/0x130
[3428822.311486]  ? bio_add_page+0x48/0x60
[3428822.311556]  vdev_disk_io_start+0x6ee/0x9c0 [zfs]
[3428822.311608]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.311668]  ? vdev_raidz_asize+0x60/0x60 [zfs]
[3428822.311710]  zio_nowait+0xad/0x140 [zfs]
[3428822.311754]  vdev_raidz_io_start+0x218/0x2d0 [zfs]
[3428822.311798]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.311854]  ? vdev_config_sync+0x230/0x230 [zfs]
[3428822.311912]  zio_nowait+0xad/0x140 [zfs]
[3428822.311963]  vdev_mirror_io_start+0x97/0x180 [zfs]
[3428822.312019]  ? spa_config_enter+0xb2/0x100 [zfs]
[3428822.312080]  zio_vdev_io_start+0x259/0x2e0 [zfs]
[3428822.312087]  ? tsd_get_by_thread+0x2a/0x40 [spl]
[3428822.312091]  ? taskq_member+0x14/0x20 [spl]
[3428822.312147]  zio_execute+0x90/0xf0 [zfs]
[3428822.312154]  taskq_thread+0x2df/0x520 [spl]
[3428822.312157]  ? wake_up_q+0x70/0x70
[3428822.312210]  ? zio_taskq_member.isra.8.constprop.13+0x70/0x70 [zfs]
[3428822.312219]  ? taskq_thread_spawn+0x50/0x50 [spl]
[3428822.312222]  kthread+0x112/0x130
[3428822.312224]  ? kthread_bind+0x30/0x30
[3428822.312227]  ret_from_fork+0x35/0x40
[3428822.312230] INFO: task z_wr_int:38076 blocked for more than 120 seconds.
[3428822.313372]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.314334] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.315247] z_wr_int        D    0 38076      2 0x80000000
[3428822.315249] Call Trace:
[3428822.315253]  __schedule+0x2a2/0x870
[3428822.315258]  schedule+0x28/0x80
[3428822.315262]  io_schedule+0x12/0x40
[3428822.315268]  wbt_wait+0x205/0x300
[3428822.315272]  ? wbt_wait+0x300/0x300
[3428822.315276]  rq_qos_throttle+0x31/0x40
[3428822.315281]  blk_mq_make_request+0x111/0x530
[3428822.315286]  generic_make_request+0x1a4/0x400
[3428822.315290]  submit_bio+0x45/0x130
[3428822.315295]  ? bio_add_page+0x48/0x60
[3428822.315343]  vdev_disk_io_start+0x6ee/0x9c0 [zfs]
[3428822.315392]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.315440]  zio_execute+0x90/0xf0 [zfs]
[3428822.315486]  vdev_queue_io_done+0x17c/0x230 [zfs]
[3428822.315532]  zio_vdev_io_done+0xc5/0x1f0 [zfs]
[3428822.315579]  zio_execute+0x90/0xf0 [zfs]
[3428822.315589]  taskq_thread+0x2df/0x520 [spl]
[3428822.315591]  ? wake_up_q+0x70/0x70
[3428822.315633]  ? zio_taskq_member.isra.8.constprop.13+0x70/0x70 [zfs]
[3428822.315638]  ? taskq_thread_spawn+0x50/0x50 [spl]
[3428822.315640]  kthread+0x112/0x130
[3428822.315642]  ? kthread_bind+0x30/0x30
[3428822.315644]  ret_from_fork+0x35/0x40
[3428822.315647] INFO: task z_wr_int:38078 blocked for more than 120 seconds.
[3428822.316532]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.317428] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.318320] z_wr_int        D    0 38078      2 0x80000000
[3428822.318322] Call Trace:
[3428822.318325]  __schedule+0x2a2/0x870
[3428822.318327]  schedule+0x28/0x80
[3428822.318329]  io_schedule+0x12/0x40
[3428822.318331]  wbt_wait+0x205/0x300
[3428822.318332]  ? wbt_wait+0x300/0x300
[3428822.318335]  rq_qos_throttle+0x31/0x40
[3428822.318337]  blk_mq_make_request+0x111/0x530
[3428822.318339]  generic_make_request+0x1a4/0x400
[3428822.318341]  submit_bio+0x45/0x130
[3428822.318343]  ? bio_add_page+0x48/0x60
[3428822.318390]  vdev_disk_io_start+0x6ee/0x9c0 [zfs]
[3428822.318434]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.318478]  zio_execute+0x90/0xf0 [zfs]
[3428822.318522]  vdev_queue_io_done+0x17c/0x230 [zfs]
[3428822.318565]  zio_vdev_io_done+0xc5/0x1f0 [zfs]
[3428822.318607]  zio_execute+0x90/0xf0 [zfs]
[3428822.318613]  taskq_thread+0x2df/0x520 [spl]
[3428822.318615]  ? wake_up_q+0x70/0x70
[3428822.318657]  ? zio_taskq_member.isra.8.constprop.13+0x70/0x70 [zfs]
[3428822.318662]  ? taskq_thread_spawn+0x50/0x50 [spl]
[3428822.318664]  kthread+0x112/0x130
[3428822.318665]  ? kthread_bind+0x30/0x30
[3428822.318667]  ret_from_fork+0x35/0x40
[3428822.318669] INFO: task z_wr_int:38079 blocked for more than 120 seconds.
[3428822.319582]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.320480] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.321399] z_wr_int        D    0 38079      2 0x80000000
[3428822.321400] Call Trace:
[3428822.321403]  __schedule+0x2a2/0x870
[3428822.321405]  schedule+0x28/0x80
[3428822.321407]  io_schedule+0x12/0x40
[3428822.321409]  wbt_wait+0x205/0x300
[3428822.321410]  ? wbt_wait+0x300/0x300
[3428822.321413]  rq_qos_throttle+0x31/0x40
[3428822.321415]  blk_mq_make_request+0x111/0x530
[3428822.321417]  generic_make_request+0x1a4/0x400
[3428822.321419]  submit_bio+0x45/0x130
[3428822.321421]  ? bio_add_page+0x48/0x60
[3428822.321466]  vdev_disk_io_start+0x6ee/0x9c0 [zfs]
[3428822.321511]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.321553]  zio_execute+0x90/0xf0 [zfs]
[3428822.321595]  vdev_queue_io_done+0x17c/0x230 [zfs]
[3428822.321639]  zio_vdev_io_done+0xc5/0x1f0 [zfs]
[3428822.321690]  zio_execute+0x90/0xf0 [zfs]
[3428822.321700]  taskq_thread+0x2df/0x520 [spl]
[3428822.321703]  ? wake_up_q+0x70/0x70
[3428822.321750]  ? zio_taskq_member.isra.8.constprop.13+0x70/0x70 [zfs]
[3428822.321756]  ? taskq_thread_spawn+0x50/0x50 [spl]
[3428822.321758]  kthread+0x112/0x130
[3428822.321759]  ? kthread_bind+0x30/0x30
[3428822.321761]  ret_from_fork+0x35/0x40
[3428822.321763] INFO: task z_wr_int:38080 blocked for more than 120 seconds.
[3428822.322664]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.323591] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.324504] z_wr_int        D    0 38080      2 0x80000000
[3428822.324506] Call Trace:
[3428822.324509]  __schedule+0x2a2/0x870
[3428822.324511]  schedule+0x28/0x80
[3428822.324512]  io_schedule+0x12/0x40
[3428822.324514]  wbt_wait+0x205/0x300
[3428822.324516]  ? wbt_wait+0x300/0x300
[3428822.324518]  rq_qos_throttle+0x31/0x40
[3428822.324520]  blk_mq_make_request+0x111/0x530
[3428822.324522]  generic_make_request+0x1a4/0x400
[3428822.324524]  submit_bio+0x45/0x130
[3428822.324526]  ? bio_add_page+0x48/0x60
[3428822.324571]  vdev_disk_io_start+0x6ee/0x9c0 [zfs]
[3428822.324615]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.324658]  zio_execute+0x90/0xf0 [zfs]
[3428822.324705]  vdev_queue_io_done+0x17c/0x230 [zfs]
[3428822.324763]  zio_vdev_io_done+0xc5/0x1f0 [zfs]
[3428822.324806]  zio_execute+0x90/0xf0 [zfs]
[3428822.324812]  taskq_thread+0x2df/0x520 [spl]
[3428822.324815]  ? wake_up_q+0x70/0x70
[3428822.324870]  ? zio_taskq_member.isra.8.constprop.13+0x70/0x70 [zfs]
[3428822.324879]  ? taskq_thread_spawn+0x50/0x50 [spl]
[3428822.324881]  kthread+0x112/0x130
[3428822.324883]  ? kthread_bind+0x30/0x30
[3428822.324885]  ret_from_fork+0x35/0x40
[3428822.324889] INFO: task z_wr_int:38081 blocked for more than 120 seconds.
[3428822.325810]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.326734] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.327670] z_wr_int        D    0 38081      2 0x80000000
[3428822.327671] Call Trace:
[3428822.327675]  __schedule+0x2a2/0x870
[3428822.327677]  schedule+0x28/0x80
[3428822.327678]  io_schedule+0x12/0x40
[3428822.327680]  wbt_wait+0x205/0x300
[3428822.327682]  ? wbt_wait+0x300/0x300
[3428822.327687]  rq_qos_throttle+0x31/0x40
[3428822.327691]  blk_mq_make_request+0x111/0x530
[3428822.327696]  generic_make_request+0x1a4/0x400
[3428822.327701]  ? bvec_alloc+0x51/0xe0
[3428822.327706]  submit_bio+0x45/0x130
[3428822.327710]  ? bio_add_page+0x48/0x60
[3428822.327757]  vdev_disk_io_start+0x6ee/0x9c0 [zfs]
[3428822.327802]  zio_vdev_io_start+0xe9/0x2e0 [zfs]
[3428822.327848]  zio_nowait+0xad/0x140 [zfs]
[3428822.327893]  vdev_queue_io_done+0x1de/0x230 [zfs]
[3428822.327938]  zio_vdev_io_done+0xc5/0x1f0 [zfs]
[3428822.327982]  zio_execute+0x90/0xf0 [zfs]
[3428822.327989]  taskq_thread+0x2df/0x520 [spl]
[3428822.327993]  ? wake_up_q+0x70/0x70
[3428822.328036]  ? zio_taskq_member.isra.8.constprop.13+0x70/0x70 [zfs]
[3428822.328042]  ? taskq_thread_spawn+0x50/0x50 [spl]
[3428822.328047]  kthread+0x112/0x130
[3428822.328051]  ? kthread_bind+0x30/0x30
[3428822.328056]  ret_from_fork+0x35/0x40
[3428822.328071] INFO: task txg_sync:38188 blocked for more than 120 seconds.
[3428822.328986]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.329892] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.330808] txg_sync        D    0 38188      2 0x80000000
[3428822.330809] Call Trace:
[3428822.330813]  __schedule+0x2a2/0x870
[3428822.330819]  schedule+0x28/0x80
[3428822.330823]  schedule_timeout+0x16b/0x390
[3428822.330833]  ? __next_timer_interrupt+0xc0/0xc0
[3428822.330837]  io_schedule_timeout+0x19/0x40
[3428822.330842]  __cv_timedwait_common+0x125/0x160 [spl]
[3428822.330848]  ? finish_wait+0x80/0x80
[3428822.330891]  zio_wait+0x12a/0x260 [zfs]
[3428822.330932]  dsl_pool_sync+0x106/0x510 [zfs]
[3428822.330978]  spa_sync+0x598/0xf60 [zfs]
[3428822.330984]  ? mutex_lock+0xe/0x30
[3428822.331046]  ? spa_txg_history_init_io+0xfe/0x110 [zfs]
[3428822.331087]  txg_sync_thread+0x294/0x460 [zfs]
[3428822.331091]  ? __switch_to_asm+0x41/0x70
[3428822.331133]  ? txg_thread_exit.isra.9+0x60/0x60 [zfs]
[3428822.331138]  ? __thread_exit+0x20/0x20 [spl]
[3428822.331142]  thread_generic_wrapper+0x6f/0x80 [spl]
[3428822.331145]  kthread+0x112/0x130
[3428822.331146]  ? kthread_bind+0x30/0x30
[3428822.331148]  ret_from_fork+0x35/0x40
[3428822.331159] INFO: task worker:2004 blocked for more than 120 seconds.
[3428822.332101]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.333006] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.333919] worker          D    0  2004      1 0x00000320
[3428822.333921] Call Trace:
[3428822.333925]  __schedule+0x2a2/0x870
[3428822.333927]  schedule+0x28/0x80
[3428822.333931]  cv_wait_common+0xfb/0x130 [spl]
[3428822.333933]  ? finish_wait+0x80/0x80
[3428822.333976]  zil_commit_impl+0x240/0xdb0 [zfs]
[3428822.334019]  zvol_request+0x224/0x3b0 [zfs]
[3428822.334023]  generic_make_request+0x1a4/0x400
[3428822.334025]  submit_bio+0x45/0x130
[3428822.334027]  ? bio_alloc_bioset+0xdc/0x220
[3428822.334029]  submit_bio_wait+0x53/0x80
[3428822.334031]  blkdev_issue_flush+0x7c/0xb0
[3428822.334040]  blkdev_fsync+0x2f/0x40
[3428822.334042]  do_fsync+0x38/0x70
[3428822.334044]  __x64_sys_fdatasync+0x13/0x20
[3428822.334049]  do_syscall_64+0x53/0x110
[3428822.334051]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[3428822.334054] RIP: 0033:0x7fa6ab1802e7
[3428822.334060] Code: Bad RIP value.
[3428822.334061] RSP: 002b:00007fa56a7fb560 EFLAGS: 00000293 ORIG_RAX: 000000000000004b
[3428822.334063] RAX: ffffffffffffffda RBX: 000000000000000b RCX: 00007fa6ab1802e7
[3428822.334064] RDX: 0000000000000000 RSI: 0000000000000008 RDI: 000000000000000b
[3428822.334064] RBP: 000055f190c95830 R08: 0000000000000000 R09: 00000000ffffffff
[3428822.334065] R10: 00007fa56a7fb660 R11: 0000000000000293 R12: 000055f18f8d7120
[3428822.334066] R13: 000055f190c94178 R14: 000055f191c50680 R15: 00007fa570000d40
[3428822.334076] INFO: task worker:40163 blocked for more than 120 seconds.
[3428822.335004]       Tainted: P           OE     4.19.0-10-amd64 #1 Debian 4.19.132-1
[3428822.335932] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3428822.336853] worker          D    0 40163      1 0x00000320
[3428822.336855] Call Trace:
[3428822.336858]  __schedule+0x2a2/0x870
[3428822.336860]  schedule+0x28/0x80
[3428822.336864]  cv_wait_common+0xfb/0x130 [spl]
[3428822.336867]  ? finish_wait+0x80/0x80
[3428822.336909]  zil_commit_impl+0x240/0xdb0 [zfs]
[3428822.336953]  zvol_request+0x224/0x3b0 [zfs]
[3428822.336956]  generic_make_request+0x1a4/0x400
[3428822.336958]  submit_bio+0x45/0x130
[3428822.336961]  ? bio_alloc_bioset+0xdc/0x220
[3428822.336962]  submit_bio_wait+0x53/0x80
[3428822.336964]  blkdev_issue_flush+0x7c/0xb0
[3428822.336966]  blkdev_fsync+0x2f/0x40
[3428822.336968]  do_fsync+0x38/0x70
[3428822.336969]  __x64_sys_fdatasync+0x13/0x20
[3428822.336971]  do_syscall_64+0x53/0x110
[3428822.336973]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[3428822.336975] RIP: 0033:0x7f3148ba32e7
[3428822.336978] Code: Bad RIP value.
[3428822.336979] RSP: 002b:00007f30ecad7560 EFLAGS: 00000293 ORIG_RAX: 000000000000004b
[3428822.336980] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007f3148ba32e7
[3428822.336980] RDX: 0000000000000000 RSI: 0000000000000008 RDI: 000000000000000d
[3428822.336981] RBP: 0000560b17922d10 R08: 0000000000000000 R09: 00000000ffffffff
[3428822.336982] R10: 00007f30ecad7660 R11: 0000000000000293 R12: 0000560b16014120
[3428822.336983] R13: 0000560b179266d8 R14: 0000560b179a2100 R15: 00007f30a8000b50
stale[bot] commented 2 years ago

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.