[Kernel panic] BUG: unable to handle kernel NULL pointer dereference

[xuzhaokui]

Running ovs-v2.3.2 on ubuntu 12.04-lts with kernel 3.13.0-55-generic, it panics like this:

[195686.613495] BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
[195686.619873] IP: [<ffffffff8169ee18>] tcp_sacktag_write_queue+0x388/0x900
[195686.626038] PGD bfa35c067 PUD bfc358067 PMD 0 
[195686.631977] Oops: 0000 [#1] SMP 
[195686.637763] Modules linked in: netconsole configfs iptable_nat nf_nat_ipv4 xt_addrtype nf_nat bridge stp llc aufs veth ip6table_filter ip6_tables iptable_filter ip_tables x_tables openvswitch vxlan ip_tunnel gre libcrc32c deflate ctr twofish_generic twofish_avx_x86_64 twofish_x86_64_3way twofish_x86_64 twofish_common camellia_generic camellia_aesni_avx_x86_64 camellia_x86_64 serpent_sse2_x86_64 glue_helper lrw serpent_generic xts gf128mul blowfish_generic blowfish_x86_64 blowfish_common cast5_avx_x86_64 ablk_helper cryptd cast5_generic cast_common des_generic cmac xcbc rmd160 crypto_null af_key xfrm_algo nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 gpio_ich dcdbas mei_me sb_edac mei edac_core lpc_ich shpchp joydev wmi acpi_power_meter ipmi_si mac_hid bonding lp parport ses enclosure hid_generic usbhid hid tg3 ptp megaraid_sas pps_core
[195686.683889] CPU: 0 PID: 3153 Comm: netagent Tainted: G        W     3.13.0-55-generic #94~precise1-Ubuntu
[195686.691138] Hardware name: Dell Inc. PowerEdge R720xd/0X6FFV, BIOS 1.6.0 03/07/2013
[195686.698524] task: ffff880bf1de3000 ti: ffff880bfbab8000 task.ti: ffff880bfbab8000
[195686.706082] RIP: 0010:[<ffffffff8169ee18>]  [<ffffffff8169ee18>] tcp_sacktag_write_queue+0x388/0x900
[195686.713852] RSP: 0018:ffff880c0fc03a58  EFLAGS: 00010a06
[195686.721645] RAX: ffff880082d23800 RBX: ffff880bf2416360 RCX: ffff880c0fc03ad8
[195686.729566] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[195686.737597] RBP: ffff880c0fc03b18 R08: 0000000000000000 R09: 0000000000000000
[195686.745587] R10: ffff880bf2416360 R11: 0000000000000000 R12: 00000000b23f1d13
[195686.753654] R13: ffff880bf2415d00 R14: ffff880bf2416360 R15: ffff880bf2415e20
[195686.761738] FS:  00007f79cdffb700(0000) GS:ffff880c0fc00000(0000) knlGS:0000000000000000
[195686.769992] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[195686.778240] CR2: 0000000000000004 CR3: 0000000bfc1eb000 CR4: 00000000000407f0
[195686.786639] Stack:
[195686.795019]  ffff880c0fc03a88 ffffffffa02740ab ffff880361482838 ffff8803f2fd7800
[195686.803764]  ffff8817fa814888 ffff880c0fc03b74 000000030fc03a01 001dca8002275741
[195686.812566]  0000000000000002 0000000000000002 01771fe4b23f2295 0000000000000000
[195686.821544] Call Trace:
[195686.830370]  <IRQ> 
[195686.830442]  [<ffffffffa02740ab>] ? vxlan_udp_encap_recv.part.44+0xbb/0x110 [vxlan]
[195686.848438]  [<ffffffffa0166a13>] ? __nf_ct_refresh_acct+0x43/0x90 [nf_conntrack]
[195686.857702]  [<ffffffff816a330e>] tcp_ack+0x54e/0x6d0
[195686.866941]  [<ffffffffa02ae544>] ? ipt_do_table+0x234/0x540 [ip_tables]
[195686.876228]  [<ffffffff816a3a9a>] tcp_rcv_established+0x2ea/0x6b0
[195686.885449]  [<ffffffffa032635e>] ? nf_nat_ipv4_fn+0x1ee/0x280 [iptable_nat]
[195686.894678]  [<ffffffff816ae1bd>] tcp_v4_do_rcv+0x15d/0x240
[195686.903848]  [<ffffffff816affd1>] tcp_v4_rcv+0x721/0x760
[195686.912954]  [<ffffffff81689dd0>] ? ip_rcv_finish+0x380/0x380
[195686.922080]  [<ffffffff8168324d>] ? nf_hook_slow+0x7d/0x150
[195686.931036]  [<ffffffff81689dd0>] ? ip_rcv_finish+0x380/0x380
[195686.939929]  [<ffffffff81689e78>] ip_local_deliver_finish+0xa8/0x220
[195686.948741]  [<ffffffff8168a17b>] ip_local_deliver+0x4b/0x90
[195686.957598]  [<ffffffff81689b71>] ip_rcv_finish+0x121/0x380
[195686.966433]  [<ffffffff8168a446>] ip_rcv+0x286/0x380
[195686.975261]  [<ffffffff81652042>] __netif_receive_skb_core+0x5e2/0x730
[195686.984180]  [<ffffffff8149b22f>] ? add_interrupt_randomness+0xef/0x140
[195686.993238]  [<ffffffff816521b1>] __netif_receive_skb+0x21/0x70
[195687.002089]  [<ffffffff81652981>] process_backlog+0xb1/0x190
[195687.010793]  [<ffffffff81653219>] net_rx_action+0x139/0x250
[195687.019442]  [<ffffffff8107035d>] __do_softirq+0xdd/0x300
[195687.028057]  [<ffffffff8107091e>] irq_exit+0x11e/0x140
[195687.036484]  [<ffffffff817720f3>] do_IRQ+0x63/0xe0
[195687.044631]  [<ffffffff8176722d>] common_interrupt+0x6d/0x6d
[195687.052580]  <EOI> 
[195687.052644]  [<ffffffff811aeda9>] ? kmem_cache_free+0x1e9/0x220
[195687.067824]  [<ffffffff8164398b>] ? skb_release_data.part.52+0xcb/0x140
[195687.075234]  [<ffffffff81641852>] ? kfree_skbmem+0x32/0xa0
[195687.082369]  [<ffffffff81766cce>] ? _raw_spin_unlock_irqrestore+0x1e/0x50
[195687.089416]  [<ffffffff810affaa>] remove_wait_queue+0x5a/0x70
[195687.096275]  [<ffffffff8154bf83>] tun_do_read.isra.24+0x103/0x180
[195687.103069]  [<ffffffff8109fc20>] ? try_to_wake_up+0x210/0x210
[195687.109697]  [<ffffffff8154c19c>] tun_chr_aio_read+0xac/0xd0
[195687.116216]  [<ffffffff811ca8ca>] do_sync_read+0x5a/0x90
[195687.122638]  [<ffffffff811cb95b>] vfs_read+0xab/0x180
[195687.128984]  [<ffffffff811cbbe2>] SyS_read+0x52/0xa0
[195687.135320]  [<ffffffff8176fddd>] system_call_fastpath+0x1a/0x1f
[195687.141723] Code: 3b 60 44 78 38 8b 51 04 eb 14 66 0f 1f 84 00 00 00 00 00 4c 39 c0 74 25 44 3b 60 44 78 1f 8b b0 cc 00 00 00 48 8b b8 d0 00 00 00 <0f> b7 74 37 04 01 f2 89 55 c4 48 8b 00 49 39 c7 75 d6 0f b6 55 
[195687.155358] RIP  [<ffffffff8169ee18>] tcp_sacktag_write_queue+0x388/0x900
[195687.162041]  RSP <ffff880c0fc03a58>
[195687.168644] CR2: 0000000000000004
[195687.192263] ---[ end trace 3306ec4515f3eef8 ]---
[195687.203457] Kernel panic - not syncing: Fatal exception in interrupt
[195687.217033] ------------[ cut here ]------------
[195687.223470] WARNING: CPU: 0 PID: 3153 at /build/buildd/linux-lts-trusty-3.13.0/arch/x86/kernel/smp.c:124 native_smp_send_reschedule+0x5e/0x60()
[195687.236984] Modules linked in: netconsole configfs iptable_nat nf_nat_ipv4 xt_addrtype nf_nat bridge stp llc aufs veth ip6table_filter ip6_tables iptable_filter ip_tables x_tables openvswitch vxlan ip_tunnel gre libcrc32c deflate ctr twofish_generic twofish_avx_x86_64 twofish_x86_64_3way twofish_x86_64 twofish_common camellia_generic camellia_aesni_avx_x86_64 camellia_x86_64 serpent_avx_x86_64 serpent_sse2_x86_64 glue_helper lrw serpent_generic xts gf128mul blowfish_generic blowfish_x86_64 blowfish_common cast5_avx_x86_64 ablk_helper cryptd cast5_generic cast_common des_generic cmac xcbc rmd160 crypto_null af_key xfrm_algo nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 gpio_ich dcdbas mei_me sb_edac mei edac_core lpc_ich shpchp joydev wmi acpi_power_meter ipmi_si mac_hid bonding lp parport ses enclosure hid_generic usbhid hid tg3 ptp megaraid_sas pps_core
[195687.291518] CPU: 0 PID: 3153 Comm: netagent Tainted: G      D W     3.13.0-55-generic #94~precise1-Ubuntu
[195687.299953] Hardware name: Dell Inc. PowerEdge R720xd/0X6FFV, BIOS 1.6.0 03/07/2013
[195687.308494]  000000000000007c ffff880c0fc033b0 ffffffff8175a7b4 000000000000e130
[195687.317159]  0000000000000000 ffff880c0fc033f0 ffffffff8106b07c ffff880c0fc131c0
[195687.325816]  0000000000000001 0000000000000000 ffff88183fc13140 ffff880c0fc13140
[195687.334451] Call Trace:
[195687.343036]  <IRQ>  [<ffffffff8175a7b4>] dump_stack+0x46/0x58
[195687.351793]  [<ffffffff8106b07c>] warn_slowpath_common+0x8c/0xc0
[195687.360611]  [<ffffffff8106b0ca>] warn_slowpath_null+0x1a/0x20
[195687.369368]  [<ffffffff8104321e>] native_smp_send_reschedule+0x5e/0x60
[195687.378146]  [<ffffffff810ad13e>] trigger_load_balance+0x17e/0x1f0
[195687.386902]  [<ffffffff8109e4df>] scheduler_tick+0xaf/0xf0
[195687.395643]  [<ffffffff8107a2e1>] update_process_times+0x61/0x80
[195687.404430]  [<ffffffff810dc2b3>] tick_sched_handle.isra.12+0x33/0x70
[195687.413235]  [<ffffffff810dc3dc>] tick_sched_timer+0x4c/0x80
[195687.422034]  [<ffffffff810930a6>] __run_hrtimer+0x76/0x230
[195687.430820]  [<ffffffff810dc390>] ? tick_nohz_handler+0xa0/0xa0
[195687.439676]  [<ffffffff81093897>] hrtimer_interrupt+0x107/0x260
[195687.448477]  [<ffffffff810460fb>] local_apic_timer_interrupt+0x3b/0x60
[195687.457279]  [<ffffffff817721b5>] smp_apic_timer_interrupt+0x45/0x60
[195687.466055]  [<ffffffff81770b1d>] apic_timer_interrupt+0x6d/0x80
[195687.474837]  [<ffffffff810c27c8>] ? console_unlock.part.12+0x218/0x260
[195687.483595]  [<ffffffff8174d279>] ? panic+0x194/0x1d7
[195687.492413]  [<ffffffff8174d1db>] ? panic+0xf6/0x1d7
[195687.501049]  [<ffffffff8176837a>] oops_end+0x14a/0x160
[195687.509680]  [<ffffffff8174c615>] no_context+0x1be/0x1cd
[195687.518287]  [<ffffffff8174c7f7>] __bad_area_nosemaphore+0x1d3/0x1f2
[195687.526958]  [<ffffffffa028bc00>] ? flow_mask_insert.isra.2+0x1d0/0x1f0 [openvswitch]
[195687.535763]  [<ffffffff8174c829>] bad_area_nosemaphore+0x13/0x15
[195687.544623]  [<ffffffff8176b2b2>] __do_page_fault+0x3c2/0x580
[195687.553497]  [<ffffffffa02833ee>] ? do_output+0x2e/0x50 [openvswitch]
[195687.562215]  [<ffffffffa02840c5>] ? do_execute_actions+0x1b5/0x220 [openvswitch]
[195687.570844]  [<ffffffffa028c456>] ? ovs_flow_tbl_lookup+0x76/0x90 [openvswitch]
[195687.579367]  [<ffffffffa028415b>] ? ovs_execute_actions+0x2b/0x30 [openvswitch]
[195687.587797]  [<ffffffffa0286a7d>] ? ovs_dp_process_received_packet+0x9d/0x130 [openvswitch]
[195687.596349]  [<ffffffff8176b48a>] do_page_fault+0x1a/0x70
[195687.604916]  [<ffffffff817675a8>] page_fault+0x28/0x30
[195687.613369]  [<ffffffff8169ee18>] ? tcp_sacktag_write_queue+0x388/0x900
[195687.621874]  [<ffffffff8169eb4f>] ? tcp_sacktag_write_queue+0xbf/0x900
[195687.630059]  [<ffffffffa02740ab>] ? vxlan_udp_encap_recv.part.44+0xbb/0x110 [vxlan]
[195687.638141]  [<ffffffffa0166a13>] ? __nf_ct_refresh_acct+0x43/0x90 [nf_conntrack]
[195687.646019]  [<ffffffff816a330e>] tcp_ack+0x54e/0x6d0
[195687.653673]  [<ffffffffa02ae544>] ? ipt_do_table+0x234/0x540 [ip_tables]
[195687.661176]  [<ffffffff816a3a9a>] tcp_rcv_established+0x2ea/0x6b0
[195687.668456]  [<ffffffffa032635e>] ? nf_nat_ipv4_fn+0x1ee/0x280 [iptable_nat]
[195687.675605]  [<ffffffff816ae1bd>] tcp_v4_do_rcv+0x15d/0x240
[195687.682513]  [<ffffffff816affd1>] tcp_v4_rcv+0x721/0x760
[195687.689162]  [<ffffffff81689dd0>] ? ip_rcv_finish+0x380/0x380
[195687.695617]  [<ffffffff8168324d>] ? nf_hook_slow+0x7d/0x150
[195687.701923]  [<ffffffff81689dd0>] ? ip_rcv_finish+0x380/0x380
[195687.708110]  [<ffffffff81689e78>] ip_local_deliver_finish+0xa8/0x220
[195687.714336]  [<ffffffff8168a17b>] ip_local_deliver+0x4b/0x90
[195687.720544]  [<ffffffff81689b71>] ip_rcv_finish+0x121/0x380
[195687.726639]  [<ffffffff8168a446>] ip_rcv+0x286/0x380
[195687.732620]  [<ffffffff81652042>] __netif_receive_skb_core+0x5e2/0x730
[195687.738647]  [<ffffffff8149b22f>] ? add_interrupt_randomness+0xef/0x140
[195687.744736]  [<ffffffff816521b1>] __netif_receive_skb+0x21/0x70
[195687.750790]  [<ffffffff81652981>] process_backlog+0xb1/0x190
[195687.756826]  [<ffffffff81653219>] net_rx_action+0x139/0x250
[195687.762822]  [<ffffffff8107035d>] __do_softirq+0xdd/0x300
[195687.768831]  [<ffffffff8107091e>] irq_exit+0x11e/0x140
[195687.774781]  [<ffffffff817720f3>] do_IRQ+0x63/0xe0
[195687.780711]  [<ffffffff8176722d>] common_interrupt+0x6d/0x6d
[195687.786680]  <EOI>  [<ffffffff811aeda9>] ? kmem_cache_free+0x1e9/0x220
[195687.792744]  [<ffffffff8164398b>] ? skb_release_data.part.52+0xcb/0x140
[195687.798847]  [<ffffffff81641852>] ? kfree_skbmem+0x32/0xa0
[195687.804930]  [<ffffffff81766cce>] ? _raw_spin_unlock_irqrestore+0x1e/0x50
[195687.811059]  [<ffffffff810affaa>] remove_wait_queue+0x5a/0x70
[195687.817145]  [<ffffffff8154bf83>] tun_do_read.isra.24+0x103/0x180
[195687.823256]  [<ffffffff8109fc20>] ? try_to_wake_up+0x210/0x210
[195687.829319]  [<ffffffff8154c19c>] tun_chr_aio_read+0xac/0xd0
[195687.835441]  [<ffffffff811ca8ca>] do_sync_read+0x5a/0x90
[195687.841562]  [<ffffffff811cb95b>] vfs_read+0xab/0x180
[195687.847669]  [<ffffffff811cbbe2>] SyS_read+0x52/0xa0
[195687.859706] ---[ end trace 3306ec4515f3eef9 ]---

[urifoox]

Did you ever find a solution? I'm running into the same issue