Empty/corrupt netflow files will crash indexing

[DeepMac]

I found that if indexer.py didn't get any flows out of an nfdump, such as when the netflow file was empty, it would crash. The following patch in netflowindexer/nfdump/indexer.py will resolve this:

@@ -10,6 +10,9 @@ ips = set() update = ips.update for line in subprocess.Popen(cmd, stdout=subprocess.PIPE).stdout:

• """ Netflow file may be empty/corrupt. --Jedi Mercer, 2/19/13 """
• if line == "No matched flows\n":
• return ips parts = line.split("|") if parts[6:9] != ['0','0','0']: #ipv6 sa = pack(">LLLL", int(parts[6]), int(parts[7]), int(parts[8]), int(parts[9]))

[JustinAzoff]

That's annoying on nfdumps part.. that message should probably go to stderr..

This fix looks like it would work, but I'm concerned that adding an if statement there might slow down the processing. It might be a better idea to just wrap the whole thing in a try/except block. I'll do some benchmarks and find out which way is faster

[DeepMac]

Well nfdump may often return no flows depending on the query used, so it's not strictly an error. It's just in this use case we want all the flows, :)

I haven't noticed any performance issues but then again it kept dying on me before I added that because it'd hit some empty nfcapd files. Because some of our sensors are erratic, :/

[JustinAzoff]

I went about fixing it in a (hopefully) more comprehensive way.. I tried testing with an actually empty file and noticed that gave a completely different message.. This should catch both of them.

[DeepMac]

Cool!