search

scroot / gopacket

Automatically exported from code.google.com/p/gopacket
BSD 3-Clause "New" or "Revised" License
0 stars 1 forks source link

gopacket/pfring: unexpected fault address 0x7fe10a545fd0 #16

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
I wrote this code and run it on machine with heavy traffic load (200 kpps and 
multiple Gbps). http://play.golang.org/p/gaNjiuLDMN

PACKET: 60 bytes, wire length 60 cap length 60 @ 2014-06-24 00:33:47.774857 
+0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=5c:5e:ab:24:0f:c0 DstMAC=90:e2:ba:49:85:c8 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 TOS=0 
Length=40 Id=21274 Flags=DF FragOffset=0 TTL=121 Protocol=TCP Checksum=3492 
SrcIP=176.15.237.106 DstIP=185.4.74.147 Options=[] Padding=[]}
- Layer 3 (20 bytes) = TCP  {Contents=[...] Payload=[] SrcPort=51920 
DstPort=80(http) Seq=3099176645 Ack=2054745823 DataOffset=5 FIN=false SYN=false 
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false Window=256 
Checksum=29130 Urgent=0 Options=[] Padding=[]}

PACKET: 90 bytes, wire length 90 cap length 90 @ 2014-06-24 00:33:47.774862 
+0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=90:e2:ba:49:85:c8 DstMAC=00:19:e2:b1:ef:c1 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 
TOS=192 Length=76 Id=27205 Flags= FragOffset=0 TTL=64 Protocol=ICMPv4 
Checksum=39893 SrcIP=195.222.9.2 DstIP=91.234.75.12 Options=[] Padding=[]}
- Layer 3 (08 bytes) = ICMPv4   {Contents=[...] Payload=[...] 
TypeCode=DestinationUnreachable(Host) Checksum=23393 Id=0 Seq=0}
- Layer 4 (48 bytes) = Payload  48 byte(s)

PACKET: 128 bytes, truncated, wire length 590 cap length 128 @ 2014-06-24 
00:33:47.774868 +0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=00:19:e2:b1:ef:c1 DstMAC=90:e2:ba:49:85:c8 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 
TOS=24 Length=576 Id=9892 Flags=DF FragOffset=0 TTL=54 Protocol=UDP 
Checksum=15117 SrcIP=192.99.19.45 DstIP=46.36.223.47 Options=[] Padding=[]}
- Layer 3 (08 bytes) = UDP  {Contents=[...] Payload=[...] SrcPort=8849 
DstPort=5060(sip) Length=556 Checksum=39383}
- Layer 4 (86 bytes) = Payload  86 byte(s)

PACKET: 90 bytes, wire length 90 cap length 90 @ 2014-06-24 00:33:47.77487 
+0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=90:e2:ba:49:85:c8 DstMAC=00:19:e2:b1:ef:c1 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 
TOS=192 Length=76 Id=31606 Flags= FragOffset=0 TTL=64 Protocol=ICMPv4 
Checksum=28815 SrcIP=195.222.9.2 DstIP=81.5.112.6 Options=[] Padding=[]}
- Layer 3 (08 bytes) = ICMPv4   {Contents=[...] Payload=[...] 
TypeCode=DestinationUnreachable(Host) Checksum=30070 Id=0 Seq=0}
- Layer 4 (48 bytes) = Payload  48 byte(s)

PACKET: 90 bytes, wire length 90 cap length 90 @ 2014-06-24 00:33:47.774872 
+0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=90:e2:ba:49:85:c8 DstMAC=00:19:e2:b1:ef:c1 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 
TOS=192 Length=76 Id=31607 Flags= FragOffset=0 TTL=64 Protocol=ICMPv4 
Checksum=28814 SrcIP=195.222.9.2 DstIP=81.5.112.6 Options=[] Padding=[]}
- Layer 3 (08 bytes) = ICMPv4   {Contents=[...] Payload=[...] 
TypeCode=DestinationUnreachable(Host) Checksum=30070 Id=0 Seq=0}
- Layer 4 (48 bytes) = Payload  48 byte(s)

PACKET: 60 bytes, wire length 60 cap length 60 @ 2014-06-24 00:33:47.774882 
+0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=90:e2:ba:49:85:c8 DstMAC=5c:5e:ab:24:0f:c0 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 TOS=0 
Length=40 Id=48437 Flags=DF FragOffset=0 TTL=63 Protocol=TCP Checksum=20186 
SrcIP=46.36.217.145 DstIP=109.104.186.162 Options=[] Padding=[]}
- Layer 3 (20 bytes) = TCP  {Contents=[...] Payload=[] SrcPort=80(http) 
DstPort=65014 Seq=60442492 Ack=3036003759 DataOffset=5 FIN=false SYN=false 
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false Window=123 
Checksum=50070 Urgent=0 Options=[] Padding=[]}

PACKET: 66 bytes, wire length 66 cap length 66 @ 2014-06-24 00:33:47.774884 
+0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=5c:5e:ab:24:0f:c0 DstMAC=90:e2:ba:49:85:c8 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 TOS=0 
Length=52 Id=59742 Flags=DF FragOffset=0 TTL=53 Protocol=TCP Checksum=10127 
SrcIP=37.144.10.85 DstIP=185.4.75.237 Options=[] Padding=[]}
- Layer 3 (32 bytes) = TCP  {Contents=[...] Payload=[] SrcPort=49727 
DstPort=80(http) Seq=554718483 Ack=2300924346 DataOffset=8 FIN=false SYN=false 
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false Window=8192 
Checksum=56362 Urgent=0 Options=[NOP, NOP, TSOPT:1035842371/1832834025] 
Padding=[]}

PACKET: 128 bytes, truncated, wire length 1514 cap length 128 @ 2014-06-24 
00:33:47.774892 +0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=90:e2:ba:49:85:c8 DstMAC=00:19:e2:b1:ef:c1 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 TOS=0 
Length=1500 Id=27441 Flags=DF FragOffset=0 TTL=63 Protocol=TCP Checksum=5691 
SrcIP=46.36.218.220 DstIP=195.225.231.205 Options=[] Padding=[]}
- Layer 3 (20 bytes) = TCP  {Contents=[...] Payload=[...] SrcPort=80(http) 
DstPort=62963 Seq=979397143 Ack=2569684808 DataOffset=5 FIN=false SYN=false 
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false Window=244 
Checksum=18320 Urgent=0 Options=[] Padding=[]}
- Layer 4 (74 bytes) = Payload  74 byte(s)

PACKET: 128 bytes, truncated, wire length 1470 cap length 128 @ 2014-06-24 
00:33:47.774894 +0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=90:e2:ba:49:85:c8 DstMAC=5c:5e:ab:24:0f:c0 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 TOS=0 
Length=1456 Id=56114 Flags=DF FragOffset=0 TTL=63 Protocol=TCP Checksum=14375 
SrcIP=159.253.18.30 DstIP=89.179.23.32 Options=[] Padding=[]}
- Layer 3 (32 bytes) = TCP  {Contents=[...] Payload=[...] SrcPort=80(http) 
DstPort=62699 Seq=2616046615 Ack=3804483635 DataOffset=8 FIN=false SYN=false 
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false Window=124 
Checksum=24274 Urgent=0 Options=[NOP, NOP, TSOPT:1968437833/893665661] 
Padding=[]}
- Layer 4 (62 bytes) = Payload  62 byte(s)

PACKET: 128 bytes, truncated, wire length 1506 cap length 128 @ 2014-06-24 
00:33:47.774895 +0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=90:e2:ba:49:85:c8 DstMAC=00:19:e2:b1:ef:c1 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 TOS=0 
Length=1492 Id=1996 Flags=DF FragOffset=0 TTL=63 Protocol=TCP Checksum=29695 
SrcIP=159.253.19.51 DstIP=5.166.1.131 Options=[] Padding=[]}
- Layer 3 (32 bytes) = TCP  {Contents=[...] Payload=[...] SrcPort=80(http) 
DstPort=58461 Seq=1902176586 Ack=3712428238 DataOffset=8 FIN=false SYN=false 
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false Window=133 
Checksum=5254 Urgent=0 Options=[NOP, NOP, TSOPT:2862413908/747611987] 
Padding=[]}
- Layer 4 (62 bytes) = Payload  62 byte(s)

unexpected fault address 0x7fe10a545fd0
throw: fault
[signal 0xb code=0x1 addr=0x7fe10a545fd0 pc=0x7fe0900613e0]

goroutine 1 [running]:
os.(*File).pread(0x7fe090ce0e18, 0x406bb0, 0x44baf9, 0xf84014e048, 0x51c038, 
...)
    /usr/lib/go/src/pkg/os/file_unix.go:170 +0x77

goroutine 2 [syscall]:
created by runtime.main
    /home/michael/DPKG/golang/src/pkg/runtime/proc.cPACKET: 66 bytes, wire length 66 cap length 66 @ 2014-06-24 00:33:47.774896 +0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=90:e2:ba:49:85:c8 DstMAC=5c:5e:ab:24:0f:c0 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 TOS=0 
Length=52 Id=54525 Flags=DF FragOffset=0 TTL=63 Protocol=TCP Checksum=12784 
SrcIP=185.4.75.237 DstIP=37.144.10.85 Options=[] Padding=[]}
- Layer 3 (32 bytes) = TCP  {Contents=[...] Payload=[] SrcPort=80(http) 
DstPort=49740 Seq=817692827 Ack=3422666756 DataOffset=8 FIN=false SYN=false 
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false Window=122 
Checksum=36559 Urgent=0 Options=[NOP, NOP, TSOPT:1832834150/1035842367] 
Padding=[]}

:221

goroutine 3 [syscall]:
code.google.com/p/gopacket/pfring._Cfunc_pfring_recv(0x2d4e2e0, 0xf84014dd10)
    /tmp/go-build365217384/code.google.com/p/gopacket/pfring/_obj/_cgo_defun.c:105 +0x2f
code.google.com/p/gopacket/pfring.(*Ring).ReadPacketDataTo(0xf84014dc30, 
0xf8425c3a00, 0x8000000080, 0x0, 0x0, ...)
    /tmp/go-build365217384/code.google.com/p/gopacket/pfring/_obj/_cgo_gotypes.go:927 +0xae
code.google.com/p/gopacket/pfring.(*Ring).ReadPacketData(0xf84014dc30, PACKET: 
60 bytes, wire length 60 cap length 60 @ 2014-06-24 00:33:47.774897 +0400 MSK
- Layer 1 (14 bytes) = Ethernet {Contents=[...] Payload=[...] 
SrcMAC=5c:5e:ab:24:0f:c0 DstMAC=90:e2:ba:49:85:c8 EthernetType=IPv4 Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[...] Payload=[...] Version=4 IHL=5 TOS=0 
Length=40 Id=14897 Flags=DF FragOffset=0 TTL=121 Protocol=TCP Checksum=38878 
SrcIP=109.104.186.162 DstIP=46.36.217.145 Options=[] Padding=[]}
- Layer 3 (20 bytes) = TCP  {Contents=[...] Payload=[] SrcPort=65013 
DstPort=80(http) Seq=4102169961 Ack=2978335724 DataOffset=5 FIN=false SYN=false 
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false 
Window=16560 Checksum=43455 Urgent=0 Options=[] Padding=[]}

0xf8425c3a00, 0x8000000080, 0x0, 0x0, ...)
    /tmp/go-build365217384/code.google.com/p/gopacket/pfring/_obj/_cgo_gotypes.go:945 +0xca
code.google.com/p/gopacket.(*PacketSource).NextPacket(0xf8417ff4b0, 
0xf841803000, 0xf840151f00, 0xf8425d2a20)
    /usr/lib/go/src/pkg/code.google.com/p/gopacket/packet.go:658 +0x57
code.google.com/p/gopacket.(*PacketSource).packetsToChannel(0xf8417ff4b0, 
0xf841803000, 0x0, 0x0)
    /usr/lib/go/src/pkg/code.google.com/p/gopacket/packet.go:675 +0x45
created by code.google.com/p/gopacket.(*PacketSource).Packets
    /usr/lib/go/src/pkg/code.google.com/p/gopacket/packet.go:695 +0x54

And without packet priting http://play.golang.org/p/E2CakSJ-7s I got:
unexpected fault address 0x7fd8dbe4c3d8
throw: fault
[signal 0xb code=0x1 addr=0x7fd8dbe4c3d8 pc=0x7fd8cd0ef3e0]

goroutine 1 [chan receive]:
main.main()
    /root/traffic_anomaly_detector/anomaly_detector.go:23 +0x1ce

goroutine 2 [syscall]:
created by runtime.main
    /home/michael/DPKG/golang/src/pkg/runtime/proc.c:221

goroutine 3 [syscall]:
code.google.com/p/gopacket/pfring._Cfunc_pfring_recv(0x3fe02e0, 0xf840245d10)
    /tmp/go-build365217384/code.google.com/p/gopacket/pfring/_obj/_cgo_defun.c:105 +0x2f
code.google.com/p/gopacket/pfring.(*Ring).ReadPacketDataTo(0xf840245c30, 
0xf841f87e00, 0x8000000080, 0x0, 0x0, ...)
    /tmp/go-build365217384/code.google.com/p/gopacket/pfring/_obj/_cgo_gotypes.go:927 +0xae
code.google.com/p/gopacket/pfring.(*Ring).ReadPacketData(0xf840245c30, 
0xf841f87e00, 0x8000000080, 0x0, 0x0, ...)
    /tmp/go-build365217384/code.google.com/p/gopacket/pfring/_obj/_cgo_gotypes.go:945 +0xca
code.google.com/p/gopacket.(*PacketSource).NextPacket(0xf8418024b0, 
0xf841806000, 0xf840144f00, 0xf841f8b900)
    /usr/lib/go/src/pkg/code.google.com/p/gopacket/packet.go:658 +0x57
code.google.com/p/gopacket.(*PacketSource).packetsToChannel(0xf8418024b0, 
0xf841806000, 0x0, 0x0)
    /usr/lib/go/src/pkg/code.google.com/p/gopacket/packet.go:675 +0x45
created by code.google.com/p/gopacket.(*PacketSource).Packets
    /usr/lib/go/src/pkg/code.google.com/p/gopacket/packet.go:695 +0x54

Original issue reported on code.google.com by pavel.odintsov on 23 Jun 2014 at 8:49

GoogleCodeExporter commented 9 years ago 
Did this happen after the fix for Issue 15 was pulled down?

Original comment by gconnell@google.com on 24 Jun 2014 at 3:49

GoogleCodeExporter commented 9 years ago 
Hello!

Yes, I'm use already fixed version:
140         data = data[:ci.CaptureLength]

Original comment by pavel.odintsov on 24 Jun 2014 at 6:37

GoogleCodeExporter commented 9 years ago 
Hey, Pavel,

I'm trying to reproduce this locally and as yet I'm not having any luck.

Can you drop me the version of PF_RING you're currently using?  Also, I'm 
wondering if maybe the pfring isn't happy about being passed between multiple 
threads.  Maybe try using runtime.LockOSThread() before your call to Packets() 
and see if that makes the issue go away?

Thanks for the patience, and hopefully we can figure this out soon.

Original comment by gsconn...@gmail.com on 26 Jun 2014 at 5:58

GoogleCodeExporter commented 9 years ago 
Hello!

I suppose this bug is fixed! I found new bug (not yours!) when PF_RING library 
does not match to PF_RING kernel module exactly. I result to unexpected 
segfaults like this.

When I upgraded module and library to same up to date version all be fine.

Memory consumption is ok too:
18526 root      20   0  130m  65m 3132 R 104.5  0.2   2:40.94 anomaly_detecto   

Thank you for you help and your excellen facility for processing packets! :)

Original comment by pavel.odintsov on 30 Jun 2014 at 8:46

GoogleCodeExporter commented 9 years ago 
But my C++ tool (maybe it's interesting for you 
https://github.com/FastVPSEestiOu/fastnetmon) with PF_RING works many time 
faster then plain gopacket! :)

18526 root      20   0  130m  65m 3132 R 104.5  0.2   5:19.80 anomaly_detecto   

13785 root      20   0  108m 4684 2380 S  38.6  0.0 137:09.55 fastnetmon      

Good way for opmitiozation :)

Original comment by pavel.odintsov on 30 Jun 2014 at 8:50

GoogleCodeExporter commented 9 years ago 
Judging from update #4, I'm going to mark this bug fixed.  Thanks again for all 
your help :)

Original comment by gsconn...@gmail.com on 4 Jul 2014 at 1:38