stale[bot]
commented
4 years ago This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.
System information
Describe the problem you're observing
GPF that in turn causes our SAN to crash requiring a hard reboot
Describe how to reproduce the problem
First we've seen it, but it was while our 2 SAN setup that establishes a MD for servers hosted over iscsi was syncing data. So, higher IO than normal, but still minimal compared to its hardware profile
Include any warning/errors/backtraces from the system logs
[ 866.483956] general protection fault: 0000 [#1] SMP [ 866.484727] Modules linked in: ib_isert target_core_user uio target_core_pscsi target_core_file target_core_iblock ip_vs nf_conntrack macvlan bonding iptable_filter ib_iser rdma_ucm ib_ucm ib_uverbs ib_umad ipmi_devintf sb_edac edac_core zfs(PO) x86_pkg_temp_thermal intel_powerclamp raid10 coretemp zunicode(PO) kvm_intel zavl(PO) zcommon(PO) znvpair(PO) kvm iTCO_wdt iTCO_vendor_support spl(O) irqbypass pcspkr joydev i2c_i801 i2c_smbus sg mei_me lpc_ich mei ioatdma shpchp mfd_core ipmi_si ipmi_msghandler acpi_power_meter acpi_pad ip_tables xfs libcrc32c mlx4_en mlx4_ib raid1 rdma_cm iw_cm ib_cm mlx5_ib ib_core sd_mod 8021q garp mrp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ast drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops [ 866.488479] mlx5_core ahci ttm igb libahci mlx4_core drm dca ptp libata pps_core i2c_algo_bit wmi sunrpc dm_mirror dm_region_hash dm_log dm_mod [ 866.489755] CPU: 8 PID: 1551 Comm: arc_reclaim Tainted: P O 4.9.32-4.el7.centos.x86_64 #1 [ 866.490414] Hardware name: Supermicro SYS-6028TP-HTFR/X10DRT-PIBF, BIOS 1.1 08/03/2015 [ 866.491090] task: ffff887f4c614b80 task.stack: ffffc900338b8000 [ 866.491812] RIP: 0010:[] [] __spl_cache_flush+0xa2/0x140 [spl]
[ 866.492511] RSP: 0018:ffffc900338bbc38 EFLAGS: 00010086
[ 866.493184] RAX: 0000000000020000 RBX: 202e746e656d6f6d RCX: 0000000000000010
[ 866.493873] RDX: 000000000001ffff RSI: 000000000001ffff RDI: ffffc9038da44018
[ 866.494573] RBP: ffffc900338bbc78 R08: ffff887cb989b578 R09: 00000001802e002a
[ 866.495281] R10: 0000000090ebf901 R11: ffff887c90ebf478 R12: ffff887f63dd2600
[ 866.496000] R13: 0000000000000040 R14: 0000000000000000 R15: ffff883f5d2c5800
[ 866.496724] FS: 0000000000000000(0000) GS:ffff887f7f200000(0000) knlGS:0000000000000000
[ 866.497463] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 866.498203] CR2: 00007fc5f41dd000 CR3: 0000000001c07000 CR4: 00000000001406e0
[ 866.498963] Stack:
[ 866.499720] 0000000800000001 ffffffffa07c3a43 ffff883f5d2c58a0 ffff883f5d2c58b8
[ 866.500486] ffff883f5d2c5800 ffff887f63dd2600 0000000000000008 0000000000000000
[ 866.501289] ffffc900338bbca8 ffffffffa07c3216 ffff887f63dd2600 ffff883f5d2c5800
[ 866.502215] Call Trace:
[ 866.503037] [] ? spl_slab_reclaim+0x53/0x210 [spl]
[ 866.503848] [] spl_cache_flush+0x36/0x50 [spl]
[ 866.504635] [] spl_kmem_cache_free+0x1ae/0x1c0 [spl]
[ 866.505457] [] zio_data_buf_free+0x5a/0x60 [zfs]
[ 866.506243] [] arc_buf_data_free.isra.16+0x1b/0x40 [zfs]
[ 866.507038] [] arc_buf_destroy+0x5a/0x170 [zfs]
[ 866.507839] [] arc_evict_state+0x41f/0x6f0 [zfs]
[ 866.508629] [] arc_adjust_impl.constprop.25+0x30/0x40 [zfs]
[ 866.509413] [] arc_adjust+0x374/0x450 [zfs]
[ 866.510181] [] arc_reclaim_thread+0xb0/0x210 [zfs]
[ 866.510938] [] ? arc_shrink+0xc0/0xc0 [zfs]
[ 866.511745] [] ? __thread_exit+0x20/0x20 [spl]
[ 866.511747] [] thread_generic_wrapper+0x72/0x80 [spl]
[ 866.511752] [] kthread+0xe6/0x100