search

stec-inc / EnhanceIO

EnhanceIO Open Source for Linux
Other
420 stars 177 forks source link

Kernel Panic On Ubuntu 14.04 Kernel 3.13 #79

Open enealDC opened 10 years ago

enealDC commented 10 years ago

Hello. My hardware details are as follows:

Power Edge R515 AMD Procs Fusion-IO Duo 640gb (latest firmware and drivers) (cache) Dell H700 RAID5 (12 15K 600GB SAS) (backing)

I'm using SCST to expose the cache backed device via SRPT to a ESX5.5 Initiator. I've mapped that LUN to an RDM disk which I'm using an ASM disk for an Oracle 11gR2.

I'm using the SLOB benchmark tool and when I'm just seeding the tables, I'm presuming after the cache gets warmed up and etc, I get a bad crash.

I can consistently reproduce this so I setup netconsole to log it to a remove machine.

http://pastebin.com/vMMpzTkS

My goal is to examine bcache, dmcache, enhancio and zfsonlinux to see if they really deliver the promise of hybrid storage in an IAAS scenario.

Any thoughts you could share would be useful

lanconnected commented 10 years ago

We've hit the same crash two times (for now) on 3.16.2-201.fc20.x86_64. Nov 21 10:08:05 10.0.12.177 kernel: [1475290.512546] ------------[ cut here ]------------ Nov 21 10:08:05 10.0.12.177 kernel: [1475290.512632] kernel BUG at /root/devel/EnhanceIO/Driver/enhanceio/eio_main.c:1522! Nov 21 10:08:05 10.0.12.177 kernel: [1475290.512775] invalid opcode: 0000 [#1] SMP Nov 21 10:08:05 10.0.12.177 kernel: [1475290.512853] Modules linked in: vhost_net vhost macvtap macvlan tun binfmt_misc drbd lru_cache ip6table_filter ip6_tables ebtable_nat ebtables bridge stp llc enhanceio_rand(OE) enhanceio_lru(OE) enhanceio_fifo(OE) enhanceio(OE) iTCO_wdt gpio_ich iTCO_vendor_su pport ppdev raid1 lpc_ich mfd_core coretemp kvm_intel kvm microcode snd_intel8x0 snd_ac97_codec serio_raw i2c_i801 ac97_bus i2c_core snd_seq snd_seq_d evice snd_pcm snd_timer snd soundcore ixgbe e1000e ptp pps_core mdio i5000_edac parport_pc edac_core parport dm_thin_pool dm_persistent_data dm_bio_pr ison libcrc32c i5k_amb ioatdma dca shpchp nfsd auth_rpcgss nfs_acl lockd sunrpc ata_generic pata_acpi arcmsr Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] CPU: 1 PID: 28242 Comm: kworker/u8:0 Tainted: G OE 3.16.2-201.fc20.x86_64 #1 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] Hardware name: Supermicro X7DA8/X7DA8, BIOS 6.00 10/02/2006 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] Workqueue: eio_callback eio_post_io_callback [enhanceio] Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] task: ffff880212378000 ti: ffff8801f0f10000 task.ti: ffff8801f0f10000 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] RIP: 0010:[] [] eio_md_write+0x24c/0x250 [enhanceio] Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] RSP: 0018:ffff8801f0f13d70 EFLAGS: 00010282 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] RAX: 0000000000000001 RBX: 000000000000e3c7 RCX: ffffffff81d0e6e0 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] RDX: 0000000000000001 RSI: 0000000000000046 RDI: 0000000000000282 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] RBP: ffff8801f0f13db0 R08: 0000000000000000 R09: 0000000000000000 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] R10: 00000000000145c0 R11: 0000000000000010 R12: ffff8801bdcb6000 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] R13: 0000000000000001 R14: 0000000000000000 R15: ffffc90010a4ee30 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] FS: 0000000000000000(0000) GS:ffff88022fc80000(0000) knlGS:0000000000000000 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] CR2: 00007fddeb7e6000 CR3: 0000000222b18000 CR4: 00000000000027e0 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] Stack: Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] ffff88021dd02000 0000000000000286 ffff8801bdcb6500 ffff8801bd01f150 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] ffff88021dd02000 ffff88018856e180 0000000000000000 0000000000e3c7d8 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] ffff8801f0f13e08 ffffffffa02381d9 0000000000000000 ffff88020000e3c7 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] Call Trace: Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] [] eio_post_io_callback+0x739/0x960 [enhanceio] Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] [] process_one_work+0x176/0x430 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] [] worker_thread+0x11d/0x530 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] [] ? rescuer_thread+0x300/0x300 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] [] kthread+0xd8/0xf0 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] [] ? insert_kthread_work+0x40/0x40 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] [] ret_from_fork+0x7c/0xb0 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] [] ? insert_kthread_work+0x40/0x40 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] Code: 48 89 72 38 48 89 42 30 49 89 57 48 49 c7 46 38 00 00 00 00 e9 05 ff ff ff 49 89 4d 38 4d 8 9 6c 24 18 e9 2a fe ff ff 0f 0b 0f 0b <0f> 0b 0f 0b 66 66 66 66 90 55 48 89 e5 41 57 49 89 ff 41 56 41 Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] RIP [] eio_md_write+0x24c/0x250 [enhanceio] Nov 21 10:08:05 10.0.12.177 kernel: [1475290.513231] RSP Nov 21 10:08:05 10.0.12.177 kernel: [1475290.517741] ---[ end trace e46e26e90cfc2532 ]---