runtime error: slice bounds out of range

[aderumier]

• Version: packetbeat 1.2.3
• Operating System: debian jessie

Aug 2 12:23:28 kvm15 /usr/bin/packetbeat[3973]: log.go:113: ParseHttp exception. Recovering, but please report this: runtime error: slice bounds out of range. Aug 2 12:23:28 kvm15 /usr/bin/packetbeat[3973]: log.go:114: Stacktrace: /go/src/github.com/elastic/beats/libbeat/logp/log.go:114 (0x4c5a96)#012/usr/local/go/src/runtime/asm_amd64.s:437 (0x47c88e)#012/usr/local/go/src/runtime/panic.go:423 (0x44b119)#012/usr/local/go/src/runtime/panic.go:18 (0x449659)#012/go/src/github.com/elastic/beats/packetbeat/protos/http/http_parser.go:165 (0x4e618c)#012/go/src/github.com/elastic/beats/packetbeat/protos/http/http_parser.go:96 (0x4e4d97)#012/go/src/github.com/elastic/beats/packetbeat/protos/http/http.go:296 (0x4df8d2)#012/go/src/github.com/elastic/beats/packetbeat/protos/http/http.go:229 (0x4df326)#012/go/src/github.com/elastic/beats/packetbeat/protos/tcp/tcp.go:101 (0x52ca31)#012/go/src/github.com/elastic/beats/packetbeat/protos/tcp/tcp.go:164 (0x52d573)#012/go/src/github.com/elastic/beats/packetbeat/decoder/decoder.go:183 (0x7137b7)#012/go/src/github.com/elastic/beats/packetbeat/decoder/decoder.go:101 (0x712cf1)#012/go/src/github.com/elastic/beats/packetbeat/sniffer/sniffer.go:356 (0x5417f9)#012/go/src/github.com/elastic/beats/packetbeat/beat/packetbeat.go:232 (0x481d4b)#012/usr/local/go/src/runtime/asm_amd64.s:1696 (0x47ebd1)

[ruflin]

Could you share some pcap files of when this happens? Can you also share your config file?

Did this happen only once or does this happen more often?

[codekoala]

I saw this while trying to gather information for a different error.

• Version: packetbeat 1.2.3 (amd64)
• OS: Arch Linux

2016/08/18 20:22:55.715837 log.go:113: ERR ParseHttp exception. Recovering, but please report this: runtime error: slice bounds out of range.
2016/08/18 20:22:55.716178 log.go:114: ERR Stacktrace: goroutine 53 [running]:
runtime/debug.Stack(0x0, 0x0, 0x0)
        /usr/lib/go/src/runtime/debug/stack.go:24 +0x80
github.com/elastic/beats/libbeat/logp.Recover(0xb591b0, 0x13)
        /build/beats/src/gopath/src/github.com/elastic/beats/libbeat/logp/log.go:114 +0x166
panic(0xa07040, 0xc82000e0c0)
        /usr/lib/go/src/runtime/panic.go:443 +0x4e9
github.com/elastic/beats/packetbeat/protos/http.(*parser).parseHTTPLine(0xc821931158, 0xc8213c7bc0, 0xc821283a00, 0x0)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/protos/http/http_parser.go:165 +0x127a
github.com/elastic/beats/packetbeat/protos/http.(*parser).parse(0xc821931158, 0xc8213c7bc0, 0xc8227b83c0)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/protos/http/http_parser.go:96 +0x67
github.com/elastic/beats/packetbeat/protos/http.(*HTTP).doParse(0xc820092100, 0xc821e27080, 0xc8227b83c0, 0xc8217aeab8, 0xc820015a00, 0xc820015ad0)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/protos/http/http.go:296 +0x3ec
github.com/elastic/beats/packetbeat/protos/http.(*HTTP).Parse(0xc820092100, 0xc8227b83c0, 0xc8217aeab8, 0xc820092100, 0x0, 0x0, 0x0, 0x0)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/protos/http/http.go:229 +0xa6
github.com/elastic/beats/packetbeat/protos/tcp.(*TcpStream).addPacket(0xc821aefac8, 0xc8227b83c0, 0xc8211f23a8)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/protos/tcp/tcp.go:101 +0x211
github.com/elastic/beats/packetbeat/protos/tcp.(*Tcp).Process(0xc8212f31a0, 0xc8211f23a8, 0xc8227b83c0)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/protos/tcp/tcp.go:164 +0x83f
github.com/elastic/beats/packetbeat/decoder.(*DecoderStruct).process(0xc8211f2000, 0xc8227b83c0, 0x2c, 0x0, 0x0)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/decoder/decoder.go:183 +0x687
github.com/elastic/beats/packetbeat/decoder.(*DecoderStruct).DecodePacketData(0xc8211f2000, 0xc821953b24, 0x408, 0x408, 0xc821aefed8)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/decoder/decoder.go:101 +0x312
github.com/elastic/beats/packetbeat/sniffer.(*SnifferSetup).Run(0xc8212f4320, 0x0, 0x0)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/sniffer/sniffer.go:356 +0xb55
github.com/elastic/beats/packetbeat/beat.(*Packetbeat).Run.func1(0xc82009e480)
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/beat/packetbeat.go:232 +0x37
created by github.com/elastic/beats/packetbeat/beat.(*Packetbeat).Run
        /build/beats/src/gopath/src/github.com/elastic/beats/packetbeat/beat/packetbeat.go:238 +0x45

I have a 90GB pcap file that I will try to narrow down to the correct window of time.

[monicasarbu]

Thank you @codekoala for your help.

[codekoala]

I used editcap to get 5 seconds before and after the timestamp of the error. I'll hold onto the 90GB pcap file for a while just in case the snippet is insufficient.

beats-2150.pcap.txt

[monicasarbu]

@codekoala What kind of traffic are you trying to monitor with Packetbeat? In the attached pcap, I see some REDIS traffic, but no HTTP. It might be that the HTTP parsing error appears when Packetbeat is trying to decode some traffic that is not HTTP. Can you please also share the configuration file with us?

[codekoala]

I've long since obliterated the environment where this was running (including the 90GB pcap dump). I was using a default configuration, only changing the output to go to logstash. Every supported input protocol was enabled, iirc.

Packetbeat was simply running on my dev box while working on a different project that used redis. Nothing particularly fancy. I wasn't looking to capture anything specifically, just to see what kind of fun packetbeat brought to my life. I was dumping the .pcap file because I stumbled upon a different error (which I brought up at https://discuss.elastic.co/t/parsememcache-udp-exception-invalid-memory-address-or-nil-pointer-dereference/57956/7) while just letting packetbeat run for a while.