Segmentation fault when creating 5 files on 4 cores

[hayley-leblanc]

Hi Rohan,

I'm running WineFS on a QEMU/KVM virtual machine with 4 CPUs, 32GB RAM, and 4GB of emulated persistent memory, running Ubuntu 20.04 and Linux 5.1.0+ with KASAN enabled. I'm finding that when I try to create 5 files or directories on a clean version of WineFS, WineFS crashes with a segmentation fault when attempting to create the 5th one. The crash appears to happen around line 1813 of inode.c, where the number of CPUs is used to calculate an ID for an inode map; I'm wondering if this computation doesn't work correctly on 4 cores? I tried creating a bunch of files on a VM with the same setup, but with 6 cores instead of 4, and didn't run into any problems. This behavior occurs in both strict and relaxed mode.

Here is the output I get upon trying to create a 5th file or directory:

[   34.687098] ==================================================================
[   34.688911] BUG: KASAN: slab-out-of-bounds in mutex_lock+0x85/0xe0
[   34.690429] Write of size 8 at addr ffff8887f8254100 by task mkdir/392
[   34.692034] 
[   34.692436] CPU: 3 PID: 392 Comm: mkdir Tainted: G            E     5.1.0+ #3
[   34.694170] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[   34.696371] Call Trace:
[   34.696987]  dump_stack+0x98/0xe4
[   34.697822]  ? mutex_lock+0x85/0xe0
[   34.698653]  print_address_description+0x71/0x239
[   34.699409]  ? mutex_lock+0x85/0xe0
[   34.699984]  ? mutex_lock+0x85/0xe0
[   34.700551]  kasan_report.cold.3+0x1b/0x3e
[   34.701210]  ? mutex_lock+0x85/0xe0
[   34.701780]  check_memory_region+0x142/0x1c0
[   34.702473]  kasan_check_write+0x18/0x20
[   34.703103]  mutex_lock+0x85/0xe0
[   34.703647]  ? __mutex_lock_slowpath+0x20/0x20
[   34.704363]  ? inode_sb_list_add+0x9c/0xd0
[   34.705024]  ? inode_init_owner+0x13d/0x160
[   34.705731]  pmfs_new_inode+0x670/0x11cd [winefs]
[   34.706502]  ? pmfs_update_inode+0x286/0x286 [winefs]
[   34.707313]  ? d_splice_alias+0x2b9/0x670
[   34.707964]  ? common_perm+0x156/0x350
[   34.708525]  pmfs_mkdir+0x1b6/0x1065 [winefs]
[   34.709006]  ? apparmor_file_receive+0xd0/0xd0
[   34.709500]  ? map_id_up+0xf9/0x1d0
[   34.709893]  ? common_perm_create.constprop.22+0x139/0x190
[   34.710510]  ? pmfs_rename+0x19fb/0x19fb [winefs]
[   34.711038]  ? security_inode_permission+0x84/0xa0
[   34.711583]  vfs_mkdir+0x21e/0x320
[   34.711968]  do_mkdirat+0x1c9/0x210
[   34.712361]  ? __ia32_sys_mknod+0x60/0x60
[   34.712813]  __x64_sys_mkdir+0x40/0x50
[   34.713236]  do_syscall_64+0x81/0x1b0
[   34.713652]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   34.714219] RIP: 0033:0x7fd2537bc687
[   34.714628] Code: 00 b8 ff ff ff ff c3 0f 1f 40 00 48 8b 05 09 d8 2b 00 64 c7 00 5f 00 00 00 b8 ff ff ff ff c3 0f 1f 40 00 b8 53 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d e1 d7 2b 00 f7 d8 64 89 08
[   34.716689] RSP: 002b:00007fff30f5cb48 EFLAGS: 00000206 ORIG_RAX: 0000000000000053
[   34.717537] RAX: ffffffffffffffda RBX: 00007fff30f5ef11 RCX: 00007fd2537bc687
[   34.718322] RDX: 0000000000000000 RSI: 00000000000001ff RDI: 00007fff30f5ef11
[   34.719070] RBP: 00007fff30f5ccc0 R08: 00000000000001ff R09: 000055c929735e10
[   34.719793] R10: 00000000000001e4 R11: 0000000000000206 R12: 0000000000000000
[   34.720515] R13: 00000000000001ff R14: 00007fff30f5ce58 R15: 0000000000000001
[   34.721240] 
[   34.721405] Allocated by task 386:
[   34.721759]  save_stack+0x43/0xd0
[   34.722107]  __kasan_kmalloc.constprop.13+0xc1/0xd0
[   34.722608]  kasan_kmalloc+0xd/0x10
[   34.722973]  __kmalloc+0x117/0x250
[   34.723334]  kmalloc_array+0x3a/0x4e [winefs]
[   34.723785]  pmfs_fill_super+0x6ae/0x2839 [winefs]
[   34.724275]  mount_bdev+0x223/0x270
[   34.724644]  pmfs_mount+0x3d/0x46 [winefs]
[   34.725063]  legacy_get_tree+0x7a/0xe0
[   34.725446]  vfs_get_tree+0x5a/0x1b0
[   34.725815]  do_mount+0xe61/0x1a00
[   34.726166]  ksys_mount+0xfe/0x110
[   34.726518]  __x64_sys_mount+0x70/0x80
[   34.726907]  do_syscall_64+0x81/0x1b0
[   34.727284]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   34.727804] 
[   34.727965] Freed by task 0:
[   34.728259] (stack is not available)
[   34.728632] 
[   34.728796] The buggy address belongs to the object at ffff8887f8254000
[   34.728796]  which belongs to the cache kmalloc-256 of size 256
[   34.730055] The buggy address is located 0 bytes to the right of
[   34.730055]  256-byte region [ffff8887f8254000, ffff8887f8254100)
[   34.731268] The buggy address belongs to the page:
[   34.731768] page:ffffea001fe09500 count:1 mapcount:0 mapping:ffff888813002fc0 index:0x0 compound_mapcount: 0
[   34.732773] flags: 0x17ffffc0010200(slab|head)
[   34.733240] raw: 0017ffffc0010200 dead000000000100 dead000000000200 ffff888813002fc0
[   34.734033] raw: 0000000000000000 0000000080190019 00000001ffffffff 0000000000000000
[   34.734821] page dumped because: kasan: bad access detected
[   34.735391] 
[   34.735556] Memory state around the buggy address:
[   34.736046]  ffff8887f8254000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   34.736780]  ffff8887f8254080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   34.737515] >ffff8887f8254100: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   34.738255]                    ^
[   34.738590]  ffff8887f8254180: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   34.739325]  ffff8887f8254200: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   34.740064] ==================================================================
[   34.740809] Disabling lock debugging due to kernel taint
[   34.741395] general protection fault: 0000 [#1] SMP KASAN
[   34.741954] CPU: 3 PID: 392 Comm: mkdir Tainted: G    B       E     5.1.0+ #3
[   34.742677] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[   34.743617] RIP: 0010:__mutex_lock.isra.6+0x5b5/0x7b0
[   34.744143] Code: be 08 00 00 00 48 89 df e8 38 18 20 ff 48 89 df e8 70 38 20 ff 48 8b 03 48 83 e0 f8 49 89 c4 74 34 48 8d 78 38 e8 4b 37 20 ff <41> 8b 44 24 38 85 c0 0f 84 31 fb ff ff 49 8d 7c 24 3c e8 34 30
[   34.746048] RSP: 0018:ffff8887ed817960 EFLAGS: 00010292
[   34.746589] RAX: 0000000000000000 RBX: ffff8887f8254100 RCX: ffffffff82288d55
[   34.747320] RDX: 0000000000000000 RSI: 0000000000000004 RDI: e3c00106000001b8
[   34.748052] RBP: ffff8887ed817ad0 R08: ffffed10ff04a821 R09: ffffed10ff04a820
[   34.748778] R10: ffffed10ff04a820 R11: 0000000000000007 R12: e3c0010600000180
[   34.749503] R13: ffff8887ec8cc440 R14: e3c0010600000182 R15: ffff8887f3e15500
[   34.750235] FS:  00007fd253ec0f40(0000) GS:ffff888813780000(0000) knlGS:0000000000000000
[   34.751062] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   34.751653] CR2: 00007fd25382babc CR3: 00000007eda62000 CR4: 00000000000006a0
[   34.752380] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   34.753105] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   34.753832] Call Trace:
[   34.754093]  ? printk+0xb0/0xd8
[   34.754421]  ? apic_timer_interrupt+0xa/0x20
[   34.754859]  ? mutex_lock+0x85/0xe0
[   34.755227]  ? ww_mutex_lock_interruptible+0xf0/0xf0
[   34.755743]  ? irq_work_claim+0x4e/0x90
[   34.756144]  ? _raw_spin_unlock_irqrestore+0x19/0x20
[   34.756656]  ? end_report+0x51/0x54
[   34.757028]  ? kasan_report.cold.3+0xe/0x3e
[   34.757463]  ? mutex_lock+0x85/0xe0
[   34.757826]  __mutex_lock_slowpath+0x13/0x20
[   34.758266]  mutex_lock+0xa6/0xe0
[   34.758611]  ? __mutex_lock_slowpath+0x20/0x20
[   34.759075]  ? inode_sb_list_add+0x9c/0xd0
[   34.759500]  ? inode_init_owner+0x13d/0x160
[   34.759950]  pmfs_new_inode+0x670/0x11cd [winefs]
[   34.760442]  ? pmfs_update_inode+0x286/0x286 [winefs]
[   34.760959]  ? d_splice_alias+0x2b9/0x670
[   34.761373]  ? common_perm+0x156/0x350
[   34.761772]  pmfs_mkdir+0x1b6/0x1065 [winefs]
[   34.762224]  ? apparmor_file_receive+0xd0/0xd0
[   34.762687]  ? map_id_up+0xf9/0x1d0
[   34.763048]  ? common_perm_create.constprop.22+0x139/0x190
[   34.763621]  ? pmfs_rename+0x19fb/0x19fb [winefs]
[   34.764108]  ? security_inode_permission+0x84/0xa0
[   34.764599]  vfs_mkdir+0x21e/0x320
[   34.764959]  do_mkdirat+0x1c9/0x210
[   34.765317]  ? __ia32_sys_mknod+0x60/0x60
[   34.765736]  __x64_sys_mkdir+0x40/0x50
[   34.766130]  do_syscall_64+0x81/0x1b0
[   34.766512]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   34.767026] RIP: 0033:0x7fd2537bc687
[   34.767397] Code: 00 b8 ff ff ff ff c3 0f 1f 40 00 48 8b 05 09 d8 2b 00 64 c7 00 5f 00 00 00 b8 ff ff ff ff c3 0f 1f 40 00 b8 53 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d e1 d7 2b 00 f7 d8 64 89 08
[   34.769299] RSP: 002b:00007fff30f5cb48 EFLAGS: 00000206 ORIG_RAX: 0000000000000053
[   34.770076] RAX: ffffffffffffffda RBX: 00007fff30f5ef11 RCX: 00007fd2537bc687
[   34.770801] RDX: 0000000000000000 RSI: 00000000000001ff RDI: 00007fff30f5ef11
[   34.771539] RBP: 00007fff30f5ccc0 R08: 00000000000001ff R09: 000055c929735e10
[   34.772269] R10: 00000000000001e4 R11: 0000000000000206 R12: 0000000000000000
[   34.772991] R13: 00000000000001ff R14: 00007fff30f5ce58 R15: 0000000000000001
[   34.773712] Modules linked in: winefs(E) bochs_drm(E) ttm(E) drm_kms_helper(E) drm(E) ppdev(E) psmouse(E) dax_pmem_compat(E) device_dax(E) pcspkr(E) fb_sys_fops(E) dax_pmem_core(E) syscopyarea(E) sysfillrect)
[   34.776816] ---[ end trace 96fcab7973ef5990 ]---
[   34.777304] RIP: 0010:__mutex_lock.isra.6+0x5b5/0x7b0
[   34.777833] Code: be 08 00 00 00 48 89 df e8 38 18 20 ff 48 89 df e8 70 38 20 ff 48 8b 03 48 83 e0 f8 49 89 c4 74 34 48 8d 78 38 e8 4b 37 20 ff <41> 8b 44 24 38 85 c0 0f 84 31 fb ff ff 49 8d 7c 24 3c e8 34 30
[   34.779752] RSP: 0018:ffff8887ed817960 EFLAGS: 00010292
[   34.780289] RAX: 0000000000000000 RBX: ffff8887f8254100 RCX: ffffffff82288d55
[   34.781015] RDX: 0000000000000000 RSI: 0000000000000004 RDI: e3c00106000001b8
[   34.781742] RBP: ffff8887ed817ad0 R08: ffffed10ff04a821 R09: ffffed10ff04a820
[   34.782466] R10: ffffed10ff04a820 R11: 0000000000000007 R12: e3c0010600000180
[   34.783216] R13: ffff8887ec8cc440 R14: e3c0010600000182 R15: ffff8887f3e15500
[   34.783957] FS:  00007fd253ec0f40(0000) GS:ffff888813780000(0000) knlGS:0000000000000000
[   34.784778] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   34.785367] CR2: 00007fd25382babc CR3: 00000007eda62000 CR4: 00000000000006a0
[   34.786094] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   34.786835] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Segmentation fault

[rohankadekodi]

Thanks for pointing this out! It has been fixed and committed in the main branch.