special config consideration using 0.0.0.0/0.0.0.0 in config file (high cpu usage)

[sleinen]

From patrick.weichmann@gmail.com on April 06, 2011 16:51:22

Hi,

We have many flow packets that arrive from a few hundred routers and the need is to duplicate some of the packets to different destinations without changing the dst ip on the routers.

so we used samplicate -p 9996 -f -S 1.1.1.1/9996 2.2.2.2/9996

but then the need came up to duplicate certain routers to additional ones or remove some from the 2 already defined.

so we started using:

samplicate -p 9996 -f -S -c /etc/samplicate.conf

samplicate.conf: 5.5.5.5:1.1.1.1/9996 2.2.2.2/9996 3.3.3.3/9996 0.0.0.0/0.0.0.0:1.1.1.1/9996 2.2.2.2/9996

Now while doing a top I saw that the cpu spikes are getting quite high from samplicate and I wonder if this is really a problem or can this be ignored.

We want to add more destination and maybe remove certain destinations from matching the 0.0.0.0 rule.

I wonder if it would be easier to specify some kind of catch all without the program having to see if the 0.0.0.0 line matches?

Original issue: http://code.google.com/p/samplicator/issues/detail?id=5

[sleinen]

From patrick.weichmann@gmail.com on April 06, 2011 07:52:24

actually I forgot to change defect to something else. Sorry.

[sleinen]

From patrick.weichmann@gmail.com on April 08, 2011 04:18:46

Our requirement is:

We don't know how many sources will send netflow (400+), i.e. we don't know each source ip.

We want certain sources not to be delivered to the standard destinations, or some source to more than the standard destinations.

I'm not a developer and I cannot fully understand the code but from a test it looks like each line in the config is treated seperately and if you have a 0.0.0.0/0.0.0.0 rule and another line that have the same destination it will actually send the packets twice?

[sleinen]

From patrick.weichmann@gmail.com on April 08, 2011 04:22:41

Just confirmed that samplicator sends the packets from source that matches 2 lines twice.

Is there anyone that could implement a change that would allow samplicator to have some catch all rule if it does not match any in the config statement || or a switch that it processes only 1 entry and then if it matches stops for that source?

[sleinen]

From simon.le...@gmail.com on April 08, 2011 15:34:03

I'm refiling this as a request for enhancement.

On your two suggestions in comment 3, the second seems to be easier to implement. Maybe a new `-1' option that breaks the loop through the source address/mask entries after the first match.

On the other hand, the first option allows some behaviors that the second doesn't, such as sending multiple copies of a packet when there are multiple matching non-catch-all rules for the packet.

A third option could be: Each per-source line in the configuration file can somehow be marked with a flag that means "if this line matches, skip the rest of the entries". I would suggest to use a "^" marker because it suggests "escape" (at least to me). I would also suggest that the "^" marker should come after the targets, i.e. at the end of each line, because that seems most intuitive to me - first do the sampling/copying, then ignore the rest of the lines.

This third option seems the most general, with semantics that are relatively easy to explain and understand, and the implementation shouldn't be too complex. I'd encourage you to work on this. Here's what would need to be modified - everything is in samplicate.c:

• struct source_context needs an additional flag, something like `escape_after_match'.
• This new flag needs to be correctly initialized to 0, like `debug' is.
• read_cf_file() needs to recognize the `escape' flag and set the new flag if it is seen.
• In samplicate(), the for() loop needs to be modified so that when a match is detected, and after executing the corresponding actions, the new flag is checked, and if found set, the loop should be exited via "break;".

I'd be happy to review such a patch.

Labels: -Type-Defect Type-Enhancement