search

liyi-ibm / linux

Linux kernel source tree
Other
0 stars 1 forks source link

kernel oops when out of memory #16

Open liyi-ibm opened 5 years ago

liyi-ibm commented 5 years ago 
Oops#1 Part1
<4>[764673.128833]   cache: sock_inode_cache(46056:container_e12_1542186274672_373368_01_003225), object size: 640, buffer size: 768, default order: 0, min order: 0
<4>[764673.128837]   node 0: slabs: 13, objs: 1105, free: 0
<4>[764673.128840]   node 8: slabs: 11, objs: 935, free: 0
<4>[764673.128845] net_ratelimit: 21 callbacks suppressed
<4>[764673.128845] socket: no more sockets
<4>[764675.797314] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[764675.797319]   cache: sock_inode_cache(45991:container_e12_1542186274672_373368_01_001929), object size: 640, buffer size: 768, default order: 0, min order: 0
<4>[764675.797321]   node 0: slabs: 16, objs: 1360, free: 0
<4>[764675.797323]   node 8: slabs: 6, objs: 510, free: 0
<4>[764675.797325] socket: no more sockets
<4>[764675.842568] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[764675.842571]   cache: sock_inode_cache(46196:container_e12_1542186274672_373368_01_002625), object size: 640, buffer size: 768, default order: 0, min order: 0
<4>[764675.842573]   node 0: slabs: 9, objs: 765, free: 0
<4>[764675.842574]   node 8: slabs: 9, objs: 765, free: 0
<4>[764675.842575] socket: no more sockets
<4>[764676.381315] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[764676.381321]   cache: anon_vma_chain(45751:container_e12_1542186274672_373368_01_003369), object size: 64, buffer size: 64, default order: 0, min order: 0
<4>[764676.381324]   node 0: slabs: 25, objs: 25600, free: 0
<4>[764676.381328]   node 8: slabs: 66, objs: 67584, free: 0
<4>[764676.905157] SLUB: Unable to allocate memory on node -1, gfp=0x14080c0(GFP_KERNEL|__GFP_ZERO)
<4>[764676.905163]   cache: anon_vma_chain(45541:container_e12_1542186274672_373368_01_000081), object size: 64, buffer size: 64, default order: 0, min order: 0
<4>[764676.905166]   node 0: slabs: 18, objs: 18432, free: 0
<4>[764676.905168]   node 8: slabs: 62, objs: 63488, free: 0
<4>[764678.126769] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[764678.126777]   cache: pid(45866:container_e12_1542186274672_373368_01_002817), object size: 80, buffer size: 128, default order: 0, min order: 0
<4>[764678.126780]   node 0: slabs: 8, objs: 4096, free: 0
<4>[764678.126782]   node 8: slabs: 20, objs: 10240, free: 0
<4>[764678.132317] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[764678.132320]   cache: task_struct(45743:container_e12_1542186274672_373368_01_002553), object size: 5504, buffer size: 5504, default order: 2, min order: 0
<4>[764678.132322]   node 0: slabs: 18, objs: 738, free: 0
<4>[764678.132323]   node 8: slabs: 8, objs: 376, free: 0
<1>[764678.251332] Unable to handle kernel paging request for data at address 0x00000004
<1>[764678.251425] Faulting instruction address: 0xc000000000207dd4
<4>[764678.251470] Oops: Kernel access of bad area, sig: 11 [#1]
<4>[764678.251497] LE SMP NR_CPUS=1024 NUMA PowerNV
<4>[764678.251527] Modules linked in: i2c_dev joydev at24 i2c_opal ofpart powernv_flash mtd ipmi_powernv ipmi_devintf ipmi_msghandler opal_prd nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc binfmt_misc usb_storage ast i2c_algo_bit drm_kms_helper ixgbe syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mpt3sas mdio drm ptp pps_core raid_class scsi_transport_sas i2c_core
<4>[764678.251704] CPU: 4 PID: 0 Comm: swapper/4 Not tainted 4.14.49-memctrl #1
<4>[764678.251737] task: c000001fee612d00 task.stack: c000001fee730000
<4>[764678.251770] NIP:  c000000000207dd4 LR: c00000000013ad58 CTR: c00000000027c620
<4>[764678.251809] REGS: c000001fee733190 TRAP: 0300   Not tainted  (4.14.49-memctrl)
<4>[764678.251847] MSR:  9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE>  CR: 44028288  XER: 00000000
<4>[764678.251890] CFAR: c00000000013ad54 DAR: 0000000000000004 DSISR: 40000000 SOFTE: 0 
<4>[764678.251890] GPR00: c00000000013ad58 c000001fee733410 c0000000013f3b00 0000000000000000 
<4>[764678.251890] GPR04: 0000000000000003 0000000000000000 c000001fee7335a8 0000000000000001 
<4>[764678.251890] GPR08: 000000000000000a c000001fee5f0480 0000000000000002 0000000000000001 
<4>[764678.251890] GPR12: 0000000000008000 c000000007d42c00 c000001fee733f90 0000000000200042 
<4>[764678.251890] GPR16: 0000000000000001 00000001048e51a1 c000001fee730000 0000000000000000 
<4>[764678.251890] GPR20: c000000000f64f80 0000000000000005 c000001fee733580 0000000000000001 
<4>[764678.251890] GPR24: c000001fee7335a8 0000000000000000 0000000000000000 0000000000000003 
<4>[764678.251890] GPR28: 0000000000000000 c000000e207d075c 0000000000000000 c000000e207d0000 
<4>[764678.252236] NIP [c000000000207dd4] __delayacct_blkio_end+0x14/0x40
<4>[764678.252271] LR [c00000000013ad58] try_to_wake_up+0x398/0x5a0
<4>[764678.252303] Call Trace:
<4>[764678.252319] [c000001fee733410] [c00000000013aa1c] try_to_wake_up+0x5c/0x5a0 (unreliable)
<4>[764678.252360] [c000001fee733490] [c00000000015d370] autoremove_wake_function+0x30/0xc0
<4>[764678.252400] [c000001fee7334d0] [c00000000027c69c] wake_page_function+0x7c/0xa0
<4>[764678.252439] [c000001fee7334f0] [c00000000015cad8] __wake_up_common+0xe8/0x240
<4>[764678.252478] [c000001fee733560] [c00000000027d16c] wake_up_page_bit+0xdc/0x150
<4>[764678.252519] [c000001fee7335e0] [c000000000477c54] mpage_end_io+0x84/0x150
<4>[764678.252553] [c000001fee733630] [c000000000542640] bio_endio+0x160/0x230
<4>[764678.252587] [c000001fee733670] [c00000000054d670] blk_update_request+0xf0/0x490
<4>[764678.252628] [c000001fee733700] [c000000000717988] scsi_end_request+0x58/0x280
<4>[764678.252667] [c000001fee733760] [c000000000717dc4] scsi_io_completion+0x214/0x6d0
<4>[764678.252706] [c000001fee733830] [c00000000070c508] scsi_finish_command+0x148/0x1b0
<4>[764678.252745] [c000001fee7338b0] [c000000000716f68] scsi_softirq_done+0x1a8/0x200
<4>[764678.252785] [c000001fee733940] [c000000000558f20] blk_done_softirq+0xe0/0x110
<4>[764678.252826] [c000001fee733980] [c000000000af9e9c] __do_softirq+0x19c/0x424
<4>[764678.252860] [c000001fee733a70] [c000000000105d28] irq_exit+0x138/0x150
<4>[764678.252894] [c000001fee733a90] [c000000000042ec0] doorbell_exception+0xb0/0xe0
<4>[764678.252935] [c000001fee733ac0] [c00000000000a2b0] h_doorbell_common+0x180/0x190
<4>[764678.252976] --- interrupt: e81 at replay_interrupt_return+0x0/0x4
<4>[764678.252976]     LR = arch_local_irq_restore.part.12+0x84/0xb0
<4>[764678.253035] [c000001fee733db0] [c000001fee733e30] 0xc000001fee733e30 (unreliable)
<4>[764678.253075] [c000001fee733dd0] [c0000000008dd678] cpuidle_enter_state+0x128/0x410
<4>[764678.253115] [c000001fee733e30] [c00000000015df9c] call_cpuidle+0x4c/0x90
<4>[764678.253147] [c000001fee733e50] [c00000000015e3b0] do_idle+0x2c0/0x370
<4>[764678.253181] [c000001fee733ec0] [c00000000015e648] cpu_startup_entry+0x38/0x50
<4>[764678.253220] [c000001fee733ef0] [c00000000004938c] start_secondary+0x4ec/0x530
<4>[764678.253260] [c000001fee733f90] [c00000000000b86c] start_secondary_prolog+0x10/0x14
<4>[764678.330877] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[764678.330881]   cache: task_struct(46156:container_e12_1542186274672_373368_01_002145), object size: 5504, buffer size: 5504, default order: 2, min order: 0
<4>[764678.330883]   node 0: slabs: 4, objs: 116, free: 0
<4>[764678.330885]   node 8: slabs: 26, objs: 1222, free: 0
<4>[764678.345382] Instruction dump:
<4>[764678.345429] 38210030 e8010010 ebe1fff8 7c0803a6 4e800020 60000000 60420000 3c4c011f 
<4>[764678.345863] 3842bd40 7c0802a6 60000000 e86309d0 <81230004> 38a30010 38c30020 792a07e1 
<4>[764678.346515] ---[ end trace 70820c9e4de555f6 ]---
Oops#1 Part1
<4>[77639.127077] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[77639.127081]   cache: task_struct(7156:container_e12_1542186274672_373368_01_004318), object size: 5504, buffer size: 5504, default order: 2, min order: 0
<4>[77639.127083]   node 0: slabs: 3, objs: 69, free: 0
<4>[77639.127084]   node 8: slabs: 25, objs: 1175, free: 0
<4>[77639.174144] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[77639.174148]   cache: dentry(7296:container_e12_1542186274672_373368_01_004337), object size: 192, buffer size: 192, default order: 0, min order: 0
<4>[77639.174150]   node 0: slabs: 80, objs: 27280, free: 0
<4>[77639.174153]   node 8: slabs: 4, objs: 1364, free: 0
<4>[77639.265085] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[77639.265089]   cache: sock_inode_cache(6936:container_e12_1542186274672_373368_01_003722), object size: 640, buffer size: 768, default order: 0, min order: 0
<4>[77639.265090]   node 0: slabs: 16, objs: 1360, free: 0
<4>[77639.265092]   node 8: slabs: 0, objs: 0, free: 0
<4>[77639.265095] socket: no more sockets
<4>[77639.271259] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[77639.271262]   cache: pid(7246:container_e12_1542186274672_373368_01_004326), object size: 80, buffer size: 128, default order: 0, min order: 0
<4>[77639.271263]   node 0: slabs: 23, objs: 11776, free: 0
<4>[77639.271265]   node 8: slabs: 10, objs: 5120, free: 0
<4>[77639.334682] SLUB: Unable to allocate memory on node -1, gfp=0x14000c0(GFP_KERNEL)
<4>[77639.334685]   cache: task_struct(7216:container_e12_1542186274672_373368_01_004312), object size: 5504, buffer size: 5504, default order: 2, min order: 0
<4>[77639.334687]   node 0: slabs: 19, objs: 821, free: 0
<4>[77639.334688]   node 8: slabs: 5, objs: 235, free: 0
<4>[77640.508507] socket: no more sockets
<1>[77640.937403] Unable to handle kernel paging request for data at address 0x00000004
<1>[77640.937538] Faulting instruction address: 0xc000000000207dd4
<4>[77640.937589] Oops: Kernel access of bad area, sig: 11 [#1]
<4>[77640.937700] LE SMP NR_CPUS=1024 NUMA PowerNV
<4>[77640.937749] Modules linked in: i2c_dev joydev at24 ofpart ipmi_powernv ipmi_devintf ipmi_msghandler powernv_flash mtd i2c_opal opal_prd nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc binfmt_misc usb_storage ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ixgbe drm mpt3sas mdio ptp pps_core raid_class scsi_transport_sas i2c_core
<4>[77640.938134] CPU: 48 PID: 36960 Comm: java Not tainted 4.14.49-4.ppc64le #1
<4>[77640.938208] task: c000001e8ce91780 task.stack: c000001ed4bb4000
<4>[77640.938316] NIP:  c000000000207dd4 LR: c00000000013ad58 CTR: c00000000027c620
<4>[77640.938394] REGS: c000001ed4bb6370 TRAP: 0300   Not tainted  (4.14.49-4.ppc64le)
<4>[77640.938451] MSR:  9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE>  CR: 42428488  XER: 00000000
<4>[77640.938517] CFAR: c00000000013ad54 DAR: 0000000000000004 DSISR: 40000000 SOFTE: 0 
<4>[77640.938517] GPR00: c00000000013ad58 c000001ed4bb65f0 c0000000013f3b00 0000000000000000 
<4>[77640.938517] GPR04: 0000000000000003 0000000000000000 c000001ed4bb6788 0000000000000001 
<4>[77640.938517] GPR08: 000000000000000a c000001dd45806c0 0000000000000002 0000000000000001 
<4>[77640.938517] GPR12: 0000000000008000 c000000007d61000 c0000000012bddb8 0000000000400840 
<4>[77640.938517] GPR16: 0000000000000001 00000001007601d7 c000001ed4bb4000 0000000000000000 
<4>[77640.938517] GPR20: c000000000f64f80 0000000000000005 c000001ed4bb6760 0000000000000001 
<4>[77640.938517] GPR24: c000001ed4bb6788 0000000000000000 0000000000000000 0000000000000003 
<4>[77640.938517] GPR28: 0000000000000000 c000001f3911075c 0000000000000000 c000001f39110000 
<4>[77640.939009] NIP [c000000000207dd4] __delayacct_blkio_end+0x14/0x40
<4>[77640.939061] LR [c00000000013ad58] try_to_wake_up+0x398/0x5a0
<4>[77640.939104] Call Trace:
<4>[77640.939125] [c000001ed4bb65f0] [c00000000013aa1c] try_to_wake_up+0x5c/0x5a0 (unreliable)
<4>[77640.939199] [c000001ed4bb6670] [c00000000015d370] autoremove_wake_function+0x30/0xc0
<4>[77640.939270] [c000001ed4bb66b0] [c00000000027c69c] wake_page_function+0x7c/0xa0
<4>[77640.939324] [c000001ed4bb66d0] [c00000000015cad8] __wake_up_common+0xe8/0x240
<4>[77640.939376] [c000001ed4bb6740] [c00000000027d16c] wake_up_page_bit+0xdc/0x150
<4>[77640.939432] [c000001ed4bb67c0] [c000000000477c54] mpage_end_io+0x84/0x150
<4>[77640.939478] [c000001ed4bb6810] [c000000000542640] bio_endio+0x160/0x230
<4>[77640.939527] [c000001ed4bb6850] [c00000000054d670] blk_update_request+0xf0/0x490
<4>[77640.939585] [c000001ed4bb68e0] [c000000000717988] scsi_end_request+0x58/0x280
<4>[77640.939639] [c000001ed4bb6940] [c000000000717dc4] scsi_io_completion+0x214/0x6d0
<4>[77640.939693] [c000001ed4bb6a10] [c00000000070c508] scsi_finish_command+0x148/0x1b0
<4>[77640.939746] [c000001ed4bb6a90] [c000000000716f68] scsi_softirq_done+0x1a8/0x200
<4>[77640.939800] [c000001ed4bb6b20] [c000000000558f20] blk_done_softirq+0xe0/0x110
<4>[77640.939859] [c000001ed4bb6b60] [c000000000af9e9c] __do_softirq+0x19c/0x424
<4>[77640.939917] [c000001ed4bb6c50] [c000000000105d28] irq_exit+0x138/0x150
<4>[77640.939965] [c000001ed4bb6c70] [c000000000042ec0] doorbell_exception+0xb0/0xe0
<4>[77640.940021] [c000001ed4bb6ca0] [c00000000000a2b0] h_doorbell_common+0x180/0x190
<4>[77640.940079] --- interrupt: e81 at _raw_spin_lock+0x78/0xc0
<4>[77640.940079]     LR = list_lru_count_one+0x44/0xd0
<4>[77640.940148] [c000001ed4bb6f90] [0000000000001569] 0x1569 (unreliable)
<4>[77640.940194] [c000001ed4bb6fc0] [c0000000002ccba4] list_lru_count_one+0x44/0xd0
<4>[77640.940249] [c000001ed4bb6ff0] [c00000000036c3f4] super_cache_count+0xa4/0x130
<4>[77640.940305] [c000001ed4bb7030] [c0000000002a1544] shrink_slab+0x1f4/0x520
<4>[77640.940353] [c000001ed4bb7180] [c0000000002a73b4] shrink_node+0x194/0x420
<4>[77640.940402] [c000001ed4bb7240] [c0000000002a7b08] do_try_to_free_pages+0x128/0x4b0
<4>[77640.940454] [c000001ed4bb72e0] [c0000000002a8284] try_to_free_mem_cgroup_pages+0x144/0x2d0
<4>[77640.940508] [c000001ed4bb7370] [c00000000034b754] try_charge+0x1b4/0x8e0
<4>[77641.033778] [c000001ed4bb7430] [c0000000003536d4] mem_cgroup_try_charge+0xa4/0x330
<4>[77641.034235] [c000001ed4bb7490] [c00000000027c05c] __add_to_page_cache_locked+0x1ac/0x480
<4>[77641.034964] [c000001ed4bb74f0] [c00000000027c3b4] add_to_page_cache_lru+0x64/0x160
<4>[77641.035387] [c000001ed4bb7540] [c000000000477ed0] ext4_mpage_readpages+0x1b0/0x980
<4>[77641.035805] [c000001ed4bb7a50] [c00000000045101c] ext4_readpages+0x4c/0x60
<4>[77641.036087] [c000001ed4bb7a70] [c0000000002970e0] __do_page_cache_readahead+0x250/0x3a0
<4>[77641.036445] [c000001ed4bb7b50] [c000000000281054] filemap_fault+0x394/0x720
<4>[77641.036828] [c000001ed4bb7bd0] [c00000000045bf7c] ext4_filemap_fault+0x4c/0x80
<4>[77641.037214] [c000001ed4bb7c00] [c0000000002d2fe8] __do_fault+0x48/0x1b0
<4>[77641.037572] [c000001ed4bb7c40] [c0000000002dd2c4] __handle_mm_fault+0x1584/0x1ee0
<4>[77641.037959] [c000001ed4bb7d20] [c0000000002ddd44] handle_mm_fault+0x124/0x230
<4>[77641.038672] [c000001ed4bb7d60] [c000000000065a0c] __do_page_fault+0x1cc/0x8c0
<4>[77641.039022] [c000001ed4bb7e30] [c00000000000b284] handle_page_fault+0x18/0x38
<4>[77641.039434] Instruction dump:
<4>[77641.039464] 38210030 e8010010 ebe1fff8 7c0803a6 4e800020 60000000 60420000 3c4c011f 
<4>[77641.039963] 3842bd40 7c0802a6 60000000 e86309d0 <81230004> 38a30010 38c30020 792a07e1 
<4>[77641.099409] ---[ end trace 16d7973a47d2e116 ]---
liyi-ibm commented 5 years ago

commit b512719f771a82180211c9a315b8a7f628832b3d Author: Tejun Heo tj@kernel.org Date: Thu Jul 26 16:37:08 2018 -0700

delayacct: fix crash in delayacct_blkio_end() after delayacct init failure
liyi-ibm commented 5 years ago

fixed by: dc9336abe3f04509aa601aa9ac37cca9fc1a1d07