search

Edgio / vflow

Enterprise Network Flow Collector (IPFIX, sFlow, Netflow)
http://www.verizonmedia.com
Apache License 2.0
1.07k stars 224 forks source link

Crash when started with a lot netflow/ipfix traffic being received #67

Closed hujiayao closed 5 years ago

hujiayao commented 5 years ago

panic: runtime error: index out of range

goroutine 143 [running]: github.com/VerizonDigital/vflow/ipfix.MemCache.getShard(0x0, 0x0, 0x0, 0x100, 0xc42095600c, 0x10, 0x10, 0x0, 0x0) /home/xxx/go/src/github.com/VerizonDigital/vflow/ipfix/memcache.go:91 +0x186 github.com/VerizonDigital/vflow/ipfix.MemCache.retrieve(0x0, 0x0, 0x0, 0x100, 0xc42095600c, 0x10, 0x10, 0x0, 0x0, 0x0, ...) /home/xxx/go/src/github.com/VerizonDigital/vflow/ipfix/memcache.go:102 +0xb4 github.com/VerizonDigital/vflow/ipfix.(Decoder).decodeSet(0xc42084ce90, 0x0, 0x0, 0x0, 0xc420922040, 0x7f03e0054e58, 0x0) /home/xxx/go/src/github.com/VerizonDigital/vflow/ipfix/decoder.go:164 +0x5e6 github.com/VerizonDigital/vflow/ipfix.(Decoder).Decode(0xc42084ce90, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /home/xxx/go/src/github.com/VerizonDigital/vflow/ipfix/decoder.go:122 +0x176 main.(IPFIX).ipfixWorker(0xc4200b0150, 0xc420090480) /home/xxx/go/src/github.com/VerizonDigital/vflow/vflow/ipfix.go:244 +0x395 main.(IPFIX).run.func1(0xc4200b0150) /home/xxx/go/src/github.com/VerizonDigital/vflow/vflow/ipfix.go:129 +0x79 created by main.(*IPFIX).run /home/xxx/go/src/github.com/VerizonDigital/vflow/vflow/ipfix.go:126 +0x188

hujiayao commented 5 years ago

mCache = ipfix.GetCache(opts.IPFIXTplCacheFile) and mCacheNF9 = netflow9.GetCache(opts.NetflowV9TplCacheFile) should be excuted before the workers are started.

mehrdadrad commented 5 years ago

It does, the log shows the shard calculated out of range! the vFlow already is under heavy load in the prod but I haven't seen this. I'll try to duplicate this issue.

hujiayao commented 5 years ago

It can be consistently reproduced by using the stress test tools provided in the project.

mehrdadrad commented 5 years ago

still i couldn't , pls let me know Go version and what OS?