search

koverstreet / bcachefs

Other
662 stars 70 forks source link

Panic: trans should be locked, unlocked by bch2_readahead #711

Closed g2p closed 1 month ago

g2p commented 1 month ago

Here is the log from scripts/decode_stacktrace.sh:

<6>[  341.400656] NFSD: Using nfsdcld client tracking operations.
<6>[  341.400663] NFSD: starting 90-second grace period (net f0000000)
<6>[  394.093776] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): starting version 1.9: disk_accounting_v2 opts=metadata_replicas=2,metadata_checksum=xxhash,data_checksum=xxhash,compression=lz4,background_compression=zstd,foreground_target=ssd,background_target=hdd,promote_target=ssd,verbose
<6>[  394.093804] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): recovering from unclean shutdown
<6>[  394.093810] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): starting journal read
<6>[  394.241950] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): journal read done on device dm-17, ret 0
<6>[  403.155577] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): journal read done on device dm-18, ret 0
<6>[  403.155627] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): journal read done, replaying entries 4598160-4598160
<6>[  403.155849] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): Journal keys: 4 read, 4 after sorting and compacting
<6>[  403.211906] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): accounting_read... done
<6>[  403.234264] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): alloc_read... done
<6>[  403.291127] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): stripes_read... done
<6>[  403.291135] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): snapshots_read... done
<6>[  403.402257] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): going read-write
<6>[  403.403530] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): journal_replay... done
<6>[  403.471643] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): resume_logged_ops... done
<6>[  403.472111] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): delete_dead_inodes... done
<6>[  403.472530] bcachefs (fff77bd8-a548-4101-b7b7-a35fa30133a8): done starting filesystem
<6>[  414.147906] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): starting version 1.9: disk_accounting_v2 opts=compression=lz4,background_compression=zstd,foreground_target=ssd,background_target=hdd,promote_target=ssd,gc_reserve_percent=5,verbose
<6>[  414.147925] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): recovering from clean shutdown, journal seq 48433524
<6>[  414.241383] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): accounting_read... done
<6>[  414.262249] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): alloc_read... done
<6>[  414.725342] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): stripes_read... done
<6>[  414.725352] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): snapshots_read... done
<6>[  414.808371] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): journal_replay... done
<6>[  414.808380] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): resume_logged_ops... done
<6>[  414.808392] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): going read-write
<6>[  414.809502] bcachefs (9986b9d8-9f64-43c3-9a1d-cd397dcf3643): done starting filesystem
<0>[ 974.524559] Kernel panic - not syncing: trans should be locked, unlocked by bch2_readahead (fs/bcachefs/fs-io-buffered.c:69 fs/bcachefs/fs-io-buffered.c:260) 
<4>[  974.524578] CPU: 10 PID: 3873 Comm: nfsd Kdump: loaded Tainted: G           OE      6.10.0-g2p #31
<4>[  974.524593] Hardware name: To Be Filled By O.E.M. X570 Phantom Gaming 4/X570 Phantom Gaming 4, BIOS P5.61 02/22/2024
<4>[  974.524607] Call Trace:
<4>[  974.524614]  <TASK>
<4>[ 974.524623] dump_stack_lvl (lib/dump_stack.c:118) 
<4>[ 974.524636] dump_stack (lib/dump_stack.c:124) 
<4>[ 974.524646] panic (kernel/panic.c:347) 
<4>[ 974.524659] ? bch2_readahead (fs/bcachefs/fs-io-buffered.c:69 fs/bcachefs/fs-io-buffered.c:260) 
<4>[ 974.524669] ? bchfs_read.isra.0 (fs/bcachefs/bcachefs.h:1189 (discriminator 3) fs/bcachefs/btree_iter.h:474 (discriminator 3) fs/bcachefs/btree_iter.h:518 (discriminator 3) fs/bcachefs/fs-io-buffered.c:161 (discriminator 3)) 
<4>[ 974.524680] bch2_trans_unlocked_error (fs/bcachefs/btree_iter.c:1426) 
<4>[ 974.524691] bch2_path_get (fs/bcachefs/btree_iter.h:334 fs/bcachefs/btree_iter.c:1708) 
<4>[ 974.524701] ? bchfs_read.isra.0 (fs/bcachefs/bcachefs.h:1189 (discriminator 3) fs/bcachefs/btree_iter.h:474 (discriminator 3) fs/bcachefs/btree_iter.h:518 (discriminator 3) fs/bcachefs/fs-io-buffered.c:161 (discriminator 3)) 
<4>[ 974.524714] bchfs_read.isra.0 (fs/bcachefs/btree_iter.h:504 (discriminator 1) fs/bcachefs/btree_iter.h:518 (discriminator 1) fs/bcachefs/fs-io-buffered.c:161 (discriminator 1)) 
<4>[ 974.524729] ? srso_return_thunk (arch/x86/lib/retpoline.S:224) 
<4>[ 974.524740] ? kmem_cache_alloc_noprof (mm/slub.c:3984 (discriminator 1) mm/slub.c:4010 (discriminator 1)) 
<4>[ 974.524765] bch2_readahead (fs/bcachefs/fs-io-buffered.c:278) 
<4>[ 974.524780] read_pages (./include/linux/pagemap.h:1343 ./include/linux/pagemap.h:1385 mm/readahead.c:166) 
<4>[ 974.524794] page_cache_ra_order (./include/linux/fs.h:858 mm/readahead.c:536) 
<4>[ 974.524810] ondemand_readahead (mm/readahead.c:670) 
<4>[ 974.524822] page_cache_sync_ra (mm/readahead.c:696) 
<4>[ 974.524834] filemap_get_pages (mm/filemap.c:2531) 
<4>[ 974.524844] ? srso_return_thunk (arch/x86/lib/retpoline.S:224) 
<4>[ 974.524854] ? __ip_queue_xmit (net/ipv4/ip_output.c:544) 
<4>[ 974.524872] filemap_splice_read (mm/filemap.c:2894) 
<4>[ 974.524883] ? ilookup5_nowait (fs/inode.c:1425 (discriminator 2)) 
<4>[ 974.524910] do_splice_read (fs/splice.c:986) 
<4>[ 974.524920] splice_direct_to_actor (fs/splice.c:1089) 
<4>[ 974.524932] ? __nfsd_setattr (fs/nfsd/vfs.c:1003) nfsd
<4>[ 974.524979] ? srso_return_thunk (arch/x86/lib/retpoline.S:224) 
<4>[ 974.524992] nfsd_splice_read (fs/nfsd/vfs.c:1063) nfsd
<4>[ 974.525039] nfsd4_encode_splice_read (fs/nfsd/nfs4xdr.c:4342) nfsd
<4>[ 974.525087] nfsd4_encode_read (fs/nfsd/nfs4xdr.c:4441) nfsd
<4>[ 974.525132] nfsd4_encode_operation (fs/nfsd/nfs4xdr.c:5758) nfsd
<4>[ 974.525176] nfsd4_proc_compound (fs/nfsd/nfs4proc.c:2802) nfsd
<4>[ 974.525220] nfsd_dispatch (fs/nfsd/nfssvc.c:1004 (discriminator 1)) nfsd
<4>[ 974.525267] svc_process (net/sunrpc/svc.c:1391 net/sunrpc/svc.c:1537) sunrpc
<4>[ 974.525325] ? nfsd_svc (fs/nfsd/nfssvc.c:961) nfsd
<4>[ 974.525369] svc_recv (net/sunrpc/svc_xprt.c:831 net/sunrpc/svc_xprt.c:888) sunrpc
<4>[ 974.525428] ? nfsd_inet6addr_event (fs/nfsd/nfssvc.c:910) nfsd
<4>[ 974.525471] nfsd (fs/nfsd/nfssvc.c:939) nfsd
<4>[ 974.525511] kthread (kernel/kthread.c:389) 
<4>[ 974.525522] ? kthread_park (kernel/kthread.c:342) 
<4>[ 974.525532] ret_from_fork (arch/x86/kernel/process.c:153) 
<4>[ 974.525542] ? kthread_park (kernel/kthread.c:342) 
<4>[ 974.525552] ret_from_fork_asm (arch/x86/entry/entry_64.S:257) 
<4>[  974.525570]  </TASK>
<0>[  974.526573] Kernel Offset: 0x8a00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

Kernel is current bcachefs/master (9bf9d5f044eb6c96b4ad9f3627990bc27ac70c15).

g2p commented 1 month ago

Possibly I was too eager to test and the fix is in the latest update (git range-diff 9bf9d5f044eb6...5ce98ee72de9e)

koverstreet commented 1 month ago

yeah I pushed to master too quickly, but spotted it in the CI and fixed it