naota / linux

Linux kernel source tree
Other
3 stars 1 forks source link

Attempt to do balance convert to raid1 (without RST) aborts transaction #64

Open kdave opened 11 months ago

kdave commented 11 months ago

4 devices, created with -d single -m single, the command btrfs balance start -mconvert=raid1 leads to transaction abort. There should be a nicer error handling.


[ 3245.166429] BTRFS info (device sdi): balance: start -mconvert=raid1 -sconvert=raid1                                                                                                                                                 [66/1378]
[ 3245.181144] BTRFS error (device sdi): zoned: profile raid1 not yet supported                                                                                                                                                                 
[ 3245.202933] BTRFS error (device sdi): zoned: profile raid1 not yet supported                                                                                                                                                                 
[ 3245.219973] BTRFS error (device sdi): zoned: profile raid1 not yet supported                                                                                                                                                                 
[ 3245.229069] ------------[ cut here ]------------                                                                                                                                                                                             
[ 3245.234954] BTRFS: Transaction aborted (error -22)                                                                                                                                                                                           
[ 3245.235186] WARNING: CPU: 6 PID: 6529 at fs/btrfs/extent-tree.c:4043 find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                                                                                                        
[ 3245.320853] CPU: 6 PID: 6529 Comm: btrfs Not tainted 6.6.0-rc4-git+ #972                                                                                                                                                                     
[ 3245.320873] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008                                                                                                                                                                       
[ 3245.320885] RIP: 0010:find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                                                                                                                                                       
[ 3245.321389] Code: 03 ff ff 84 c0 75 14 f6 05 88 16 22 00 01 75 27 41 b8 01 00 00 00 e9 5b ff ff ff 44 89 e6 48 c7 c7 00 83 14 c1 e8 81 7d 20 ec <0f> 0b 41 b8 01 00 00 00 e9 3f ff ff ff 4c 89 ef e8 1c 2c 66 ec 49                          
[ 3245.321407] RSP: 0018:ffff88810671f0b0 EFLAGS: 00010282                                                                                                                                                                                      
[ 3245.361249] RAX: 0000000000000000 RBX: ffff88814141c000 RCX: 0000000000000027                                                                                                                                                                
[ 3245.361261] RDX: 0000000000000027 RSI: 0000000000000004 RDI: ffff8881e6fee888                                                                                                                                                                
[ 3245.361276] RBP: 0000000000000001 R08: ffffffffad3282fe R09: ffffed103cdfdd11                                                                                                                                                                
[ 3245.361287] R10: ffff8881e6fee88b R11: 00000000322d2072 R12: 00000000ffffffea                                                                                                                                                                
[ 3245.396212] R13: ffff88813b332568 R14: ffff88813b332518 R15: ffff88810c391000                                                                                                                                                                
[ 3245.396225] FS:  00007ff2ea7cc9c0(0000) GS:ffff8881e6e00000(0000) knlGS:0000000000000000                                                                                                                                                     
[ 3245.396237] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                                                                                                                                                                                
[ 3245.396253] CR2: 00007fd7a946c040 CR3: 00000001384f8000 CR4: 00000000000006e0                                                                                                                                                                
[ 3245.396266] Call Trace:                                                                                                                                                                                                                      
[ 3245.396280]  <TASK>                                                                                                                                                                                                                          
[ 3245.396293]  ? __warn+0xa1/0x210                                                                                                                                                                                                             
[ 3245.433595]  ? find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                                                                                                                                                              
[ 3245.434050]  ? __report_bug+0x11b/0x130                                                                                                                                                                                                      
[ 3245.434090]  ? find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                                                                                                                                                              
[ 3245.434564]  ? report_bug+0x3f/0x90                                                                    
[ 3245.434592]  ? handle_bug+0x3c/0x70                                                                                                                                                                               
[ 3245.458227]  ? exc_invalid_op+0x14/0x40                                                                                                                                                                           
[ 3245.458250]  ? asm_exc_invalid_op+0x16/0x20                                                            
[ 3245.458300]  ? irq_work_claim+0x1e/0x40                                                                                                                                                                           
[ 3245.470744]  ? find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                                                                                                                                   
[ 3245.477367]  find_free_extent+0xb4d/0x13d0 [btrfs]                                                                                                                                                                
[ 3245.477910]  ? find_free_extent_clustered+0x550/0x550 [btrfs]                                                                                                                                                     
[ 3245.478384]  ? do_raw_spin_unlock+0x93/0xf0                                                                                                                                                                       
[ 3245.478410]  ? _raw_spin_unlock+0x1f/0x40                                                              
[ 3245.478450]  btrfs_reserve_extent+0x1fd/0x380 [btrfs]                 
[ 3245.478929]  ? walk_down_tree+0x2b0/0x2b0 [btrfs]                                                      
[ 3245.479452]  ? btrfs_reserve_metadata_bytes+0x3d/0x1c0 [btrfs]                                                                                                                                                    
[ 3245.479939]  btrfs_alloc_tree_block+0x194/0x590 [btrfs]                                                
[ 3245.480427]  ? btrfs_alloc_logged_file_extent+0x250/0x250 [btrfs]                                      
[ 3245.480854]  ? mark_usage+0x90/0x190                                                                   
[ 3245.480886]  ? mark_usage+0xa0/0x190                                                                   
[ 3245.480974]  ? lock_acquire.part.0+0xf1/0x2a0                                                          
[ 3245.481001]  ? __btrfs_tree_lock+0x3c/0x150 [btrfs]                          
[ 3245.546021]  __btrfs_cow_block+0x26f/0xc60 [btrfs]                                                     
[ 3245.546570]  ? update_ref_for_cow+0x5b0/0x5b0 [btrfs]                                   
[ 3245.557116]  ? btrfs_qgroup_add_swapped_blocks+0x4e0/0x4e0 [btrfs]
[ 3245.557602]  ? btrfs_root_node+0xb0/0x260 [btrfs]                                                      
[ 3245.569229]  btrfs_cow_block+0x20b/0x3a0 [btrfs]
[ 3245.569685]  ? btrfs_search_slot_get_root+0x1f7/0x3c0 [btrfs]                                                        
[ 3245.580784]  btrfs_search_slot+0x919/0xf80 [btrfs]
[ 3245.581232]  ? do_syscall_64+0x3d/0x90                   
[ 3245.581274]  ? _raw_spin_unlock_irqrestore+0x2d/0x50                                                                 
[ 3245.581315]  ? balance_level+0x12d0/0x12d0 [btrfs]       
[ 3245.581740]  ? set_track_prepare+0x40/0x60               
[ 3245.581761]  ? __insert_orphan_inode+0x1a/0x100 [btrfs]                                                              
[ 3245.582229]  ? create_reloc_inode+0x1b8/0x2a0 [btrfs]                                                                
[ 3245.616386]  ? btrfs_relocate_chunk+0x81/0x220 [btrfs]                                                               
[ 3245.616837]  ? btrfs_balance+0x841/0xcb0 [btrfs]         
[ 3245.627242]  ? btrfs_ioctl_balance+0x205/0x410 [btrfs]                                                               
[ 3245.627687]  ? btrfs_ioctl+0x312/0xb80 [btrfs]           
[ 3245.638063]  ? __x64_sys_ioctl+0xbd/0xe0                 
[ 3245.638081]  ? do_syscall_64+0x3d/0x90
[ 3245.638102]  ? entry_SYSCALL_64_after_hwframe+0x63/0xcd                                                              
[ 3245.638152]  btrfs_insert_empty_items+0x65/0xb0 [btrfs]                                                              
[ 3245.657044]  btrfs_insert_empty_inode+0xcb/0x110 [btrfs]                                                             
[ 3245.657502]  ? btrfs_insert_inode_ref+0x460/0x460 [btrfs]                                                            
[ 3245.668927]  ? lock_is_held_type+0x9d/0x110              
[ 3245.668966]  ? kasan_save_stack+0x20/0x40                
[ 3245.669002]  ? rcu_is_watching+0x1f/0x40                 
[ 3245.681721]  ? __might_resched+0x20f/0x310               
[ 3245.681764]  __insert_orphan_inode+0x37/0x100 [btrfs]                                                                
[ 3245.691349]  create_reloc_inode+0x1b8/0x2a0 [btrfs]                                                                  
[ 3245.691818]  ? replace_path+0x1230/0x1230 [btrfs]        
[ 3245.702100]  ? kasan_quarantine_put+0x7d/0x1d0           
[ 3245.702133]  ? kasan_quarantine_put+0x7d/0x1d0           
[ 3245.702164]  ? btrfs_relocate_block_group+0x1c7/0x600 [btrfs]                                                        
[ 3245.717385]  ? kmem_cache_free+0x1b4/0x3a0               
[ 3245.717430]  btrfs_relocate_block_group+0x252/0x600 [btrfs]                                                          
[ 3245.727595]  ? lock_is_held_type+0x9d/0x110              
[ 3245.727637]  btrfs_relocate_chunk+0x81/0x220 [btrfs]                                                                 
[ 3245.737255]  __btrfs_balance+0x454/0x6c0 [btrfs]         
[ 3245.737772]  ? btrfs_relocate_sys_chunks+0x2b0/0x2b0 [btrfs]                                                         
[ 3245.748591]  ? __mutex_unlock_slowpath+0x19f/0x290       
[ 3245.748617]  ? bit_wait_io_timeout+0xd0/0xd0             
[ 3245.748675]  ? btrfs_balance+0x7e8/0xcb0 [btrfs]         
[ 3245.762966]  btrfs_balance+0x841/0xcb0 [btrfs]           
[ 3245.763474]  btrfs_ioctl_balance+0x205/0x410 [btrfs]                                                                 
[ 3245.773667]  ? btrfs_ioctl_dev_replace+0x220/0x220 [btrfs]                                                           
[ 3245.774127]  ? local_clock_noinstr+0x13/0xc0             
[ 3245.774150]  ? rcu_is_watching+0x1f/0x40                 
[ 3245.774222]  ? do_vfs_ioctl+0x465/0x980                  
[ 3245.774262]  btrfs_ioctl+0x312/0xb80 [btrfs]             
[ 3245.797375]  ? btrfs_ioctl_get_supported_features+0x30/0x30 [btrfs]                                                  
[ 3245.797849]  ? lockdep_hardirqs_on_prepare.part.0+0xf1/0x1c0                                                         
[ 3245.797875]  ? percpu_counter_add_batch+0x67/0xb0        
[ 3245.797902]  ? mark_held_locks+0x23/0x90                 
[ 3245.797941]  ? lockdep_hardirqs_on_prepare.part.0+0xf1/0x1c0                                                         
[ 3245.797968]  ? __call_rcu_common.constprop.0+0x230/0x3c0                                                             
[ 3245.798030]  __x64_sys_ioctl+0xbd/0xe0                   
[ 3245.798070]  do_syscall_64+0x3d/0x90                     
[ 3245.838636]  entry_SYSCALL_64_after_hwframe+0x63/0xcd                                                                
[ 3245.838654] RIP: 0033:0x7ff2e84e86c7                     
[ 3245.866962] RSP: 002b:00007ffefc528a48 EFLAGS: 00000246 ORIG_RAX: 0000000000000010                                   
[ 3245.866986] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007ff2e84e86c7                                        
[ 3245.866997] RDX: 00007ffefc528ae0 RSI: 00000000c4009420 RDI: 0000000000000003                                        
[ 3245.867014] RBP: 00007ffefc529823 R08: 00000000004d4123 R09: 0000000000000013                                        
[ 3245.867024] R10: 0000000022494966 R11: 0000000000000246 R12: 0000000000000001                                        
[ 3245.867035] R13: 00007ffefc528ae0 R14: 0000000000000000 R15: 00000000004dc4d2                                        
[ 3245.913543]  </TASK>                                     
[ 3245.915931] irq event stamp: 10971                       
[ 3245.915941] hardirqs last  enabled at (10977): [<ffffffffad1b9485>] console_trylock_spinning+0x175/0x190                                                                                                                                     
[ 3245.915960] hardirqs last disabled at (10982): [<ffffffffad1b9490>] console_trylock_spinning+0x180/0x190                                                                                                                                     
[ 3245.915977] softirqs last  enabled at (10872): [<ffffffffadfda473>] __do_softirq+0x2e3/0x542                         
[ 3245.915995] softirqs last disabled at (10867): [<ffffffffad0dbf15>] irq_exit_rcu+0x95/0xc0                           
[ 3245.916011] ---[ end trace 0000000000000000 ]---         
[ 3245.916406] BTRFS: error (device sdi: state A) in find_free_extent_update_loop:4043: errno=-22 unknown               
[ 3245.916433] BTRFS info (device sdi: state EA): forced readonly                                                       
[ 3245.916686] BTRFS info (device sdi: state EA): balance: ended with status: -22
kdave commented 11 months ago

And because the balance item is already stored umount/mount will instantly go read-only:

[ 4038.576986] BTRFS info (device sdi): using crc32c (crc32c-generic) checksum algorithm                                                                                                                                                                                          
[ 4038.586067] BTRFS info (device sdi): using free space tree                                                                                                                                                                                                                     
[ 4038.632560] BTRFS info (device sdi): host-managed zoned block device /dev/sdi, 40 zones of 268435456 bytes                                                                                                                                                                     
[ 4038.644614] BTRFS info (device sdi): host-managed zoned block device /dev/sdj, 40 zones of 268435456 bytes                                                                                                                                                                     
[ 4038.656432] BTRFS info (device sdi): host-managed zoned block device /dev/sdk, 40 zones of 268435456 bytes                                                                                                                                                                     
[ 4038.667879] BTRFS info (device sdi): host-managed zoned block device /dev/sdl, 40 zones of 268435456 bytes                                                                                                                                                                     
[ 4038.678744] BTRFS info (device sdi): zoned mode enabled with zone size 268435456                                                                                                                                                                                               
[ 4038.723594] BTRFS error (device sdi): zoned: profile raid1 not yet supported                                                                                                                                                                                                   
[ 4038.733609] ------------[ cut here ]------------                                                                                                                                                                                                                               
[ 4038.739601] BTRFS: Transaction aborted (error -22)                                                                                                                                                                                                                             
[ 4038.739847] WARNING: CPU: 7 PID: 6833 at fs/btrfs/extent-tree.c:4043 find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                                                                                                                                          
[ 4038.827205] CPU: 7 PID: 6833 Comm: btrfs-balance Tainted: G        W          6.6.0-rc4-git+ #972                                                                                                                                                                              
[ 4038.827235] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008                                                                                                                                                                                                         
[ 4038.827255] RIP: 0010:find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                                                                                                                                                                                         
[ 4038.850727] RSP: 0018:ffff8881175d7428 EFLAGS: 00010282                                                                                                                                                                                                                        
[ 4038.850756] RAX: 0000000000000000 RBX: ffff8881214f0000 RCX: 0000000000000027                                                                                                                                                                                                  
[ 4038.882853] RDX: 0000000000000027 RSI: 0000000000000004 RDI: ffff8881e73ee888                                                                                                                                                                                                  
[ 4038.882865] RBP: 0000000000000001 R08: ffffffffad3282fe R09: ffffed103ce7dd11                                                                                                                                                                                                  
[ 4038.882876] R10: ffff8881e73ee88b R11: 00000000322d2072 R12: 00000000ffffffea                                                                                                                                                                                                  
[ 4038.882888] R13: ffff8881505e6eb0 R14: ffff8881505e6e60 R15: ffff888111a9c000                                                                                                                                                                                                  
[ 4038.882902] FS:  0000000000000000(0000) GS:ffff8881e7200000(0000) knlGS:0000000000000000                                                                                                                                                                                       
[ 4038.882914] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                                                                                                                                                                                                                  
[ 4038.926752] CR2: 00007f30a6674040 CR3: 0000000106cfd000 CR4: 00000000000006e0                                                                                                                                                                                                  
[ 4038.926764] Call Trace:                                                                                                                                                                                                                                                        
[ 4038.926780]  <TASK>                                                                                                                                                                                                                                                            
[ 4038.926793]  ? __warn+0xa1/0x210                                                                                                                                                                                                                                               
[ 4038.942504]  ? find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                                                                                                                                                                                                
[ 4038.942956]  ? __report_bug+0x11b/0x130                                                                                                                                                                                                                                        
[ 4038.942991]  ? find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                        
[ 4038.943395]  ? report_bug+0x3f/0x90                                                                    
[ 4038.963404]  ? handle_bug+0x3c/0x70                                                                    
[ 4038.963424]  ? exc_invalid_op+0x14/0x40                                                                
[ 4038.971170]  ? asm_exc_invalid_op+0x16/0x20                                                                                           
[ 4038.971221]  ? irq_work_claim+0x1e/0x40                                                                
[ 4038.979698]  ? find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                        
[ 4038.986309]  ? find_free_extent_update_loop+0x48f/0x4c0 [btrfs]                                        
[ 4038.986782]  find_free_extent+0xb4d/0x13d0 [btrfs]                                                                                                                                                                
[ 4038.987311]  ? find_free_extent_clustered+0x550/0x550 [btrfs]                                                                                                                                                     
[ 4038.987745]  ? do_raw_spin_unlock+0x93/0xf0                                                                                                                                                                       
[ 4038.987784]  ? _raw_spin_unlock+0x1f/0x40                                                                                                                                                                         
[ 4038.987840]  btrfs_reserve_extent+0x1fd/0x380 [btrfs]                                                  
[ 4038.988321]  ? walk_down_tree+0x2b0/0x2b0 [btrfs]                                                      
[ 4038.988747]  ? __lock_release+0x14d/0x530                                                              
[ 4038.988843]  ? btrfs_use_block_rsv+0x146/0x350 [btrfs]                                                 
[ 4038.989312]  ? do_raw_spin_unlock+0x93/0xf0                                                                                                                                                                       
[ 4038.989342]  ? _raw_spin_unlock+0x1f/0x40                                                                                                                                                                         
[ 4038.989383]  btrfs_alloc_tree_block+0x194/0x590 [btrfs]
[ 4038.989892]  ? btrfs_alloc_logged_file_extent+0x250/0x250 [btrfs]                                                                                                                                                 
[ 4038.990350]  ? mark_usage+0x90/0x190                                                                                                                                                                              
[ 4038.990379]  ? mark_usage+0xa0/0x190                                                                   
[ 4039.063547]  ? lock_acquire.part.0+0xf1/0x2a0                    
[ 4039.063569]  ? __btrfs_tree_lock+0x3c/0x150 [btrfs]                                                                                   
[ 4039.073329]  __btrfs_cow_block+0x26f/0xc60 [btrfs]               
[ 4039.073860]  ? update_ref_for_cow+0x5b0/0x5b0 [btrfs]                                                                                 
[ 4039.084435]  ? btrfs_qgroup_add_swapped_blocks+0x4e0/0x4e0 [btrfs]                                                                    
[ 4039.084894]  ? btrfs_root_node+0xb0/0x260 [btrfs]                
[ 4039.096554]  btrfs_cow_block+0x20b/0x3a0 [btrfs]                 
[ 4039.096998]  ? btrfs_search_slot_get_root+0x1f7/0x3c0 [btrfs]                                                                         
[ 4039.108133]  btrfs_search_slot+0x919/0xf80 [btrfs]               
[ 4039.108625]  ? balance_level+0x12d0/0x12d0 [btrfs]               
[ 4039.109025]  ? join_transaction+0x75a/0x830 [btrfs]                                                                                   
[ 4039.109442]  ? btrfs_record_root_in_trans+0x3f/0xd0 [btrfs]                                                                           
[ 4039.130544]  btrfs_insert_empty_items+0x65/0xb0 [btrfs]                                                                               
[ 4039.130986]  insert_balance_item.isra.0+0x126/0x700 [btrfs]                                                                           
[ 4039.142477]  ? gather_device_info+0x3d0/0x3d0 [btrfs]                                                                                 
[ 4039.142892]  ? mark_usage+0x190/0x190                            
[ 4039.142915]  ? balance_kthread+0xc5/0x280 [btrfs]                
[ 4039.157315]  ? __lock_acquired+0x1f1/0x3d0                       
[ 4039.157374]  ? mark_lock.part.0+0xdd/0xa40                       
[ 4039.166235]  ? mark_lock_irq+0x5b0/0x5b0                         
[ 4039.166262]  ? lock_downgrade+0x90/0x90                          
[ 4039.166279]  ? lock_acquired+0xb3/0x180                          
[ 4039.166297]  ? rcu_is_watching+0x1f/0x40                         
[ 4039.182628]  ? mark_held_locks+0x6b/0x90                         
[ 4039.182689]  btrfs_balance+0x774/0xcb0 [btrfs]                   
[ 4039.191528]  ? btrfs_balance+0xcb0/0xcb0 [btrfs]                 
[ 4039.191967]  balance_kthread+0xe7/0x280 [btrfs]                  
[ 4039.201834]  kthread+0x19e/0x1e0                                 
[ 4039.201851]  ? kthread+0xf3/0x1e0                                
[ 4039.201865]  ? kthread_complete_and_exit+0x20/0x20               
[ 4039.213814]  ret_from_fork+0x2d/0x50                             
[ 4039.213830]  ? kthread_complete_and_exit+0x20/0x20               
[ 4039.213852]  ret_from_fork_asm+0x11/0x20                         
[ 4039.226811]  </TASK>                                             
[ 4039.226823] irq event stamp: 1775                                
[ 4039.226831] hardirqs last  enabled at (1781): [<ffffffffad1b9485>] console_trylock_spinning+0x175/0x190                               
[ 4039.226853] hardirqs last disabled at (1786): [<ffffffffad1b9490>] console_trylock_spinning+0x180/0x190                               
[ 4039.226873] softirqs last  enabled at (1686): [<ffffffffadfda473>] __do_softirq+0x2e3/0x542                                           
[ 4039.226899] softirqs last disabled at (1681): [<ffffffffad0dbf15>] irq_exit_rcu+0x95/0xc0                                             
[ 4039.226917] ---[ end trace 0000000000000000 ]---                 
[ 4039.227389] BTRFS: error (device sdi: state A) in find_free_extent_update_loop:4043: errno=-22 unknown                                
[ 4039.227438] BTRFS info (device sdi: state EA): forced readonly
kdave commented 11 months ago

Fixable by mounting with -o skip_balance and btrfs balance cancel . but still.

xuefer commented 2 weeks ago

I can to it convert to raid1. you may want to retry it. until you may hit other bug