Improve usefulness of «jool --pool4 --count» for monitoring

[toreanderson]

After the IPv4 pool was changed to accept port ranges and MARKs, the output from jool --pool4 --count changed and now shows:

$ jool -4 --count
tables: 3
samples: 3
transport addresses: 196606

This isn't particularly useful for monitoring the utilisation of the pools, because it lumps TCP/UDP/ICMP into one big bucket. But in reality it's not a big shared bucket. That is, you can be completely out of, say, TCP transport addresses even though the sum of what's reported by jool -b --count is much lower than the number of transport addresses reported above.

So I'd like to suggest making the output from jool -4 --count be a bit more verbose. That is, for each individual IPv4 pool (as distinguished by --mark) report the number of transport identifiers for each protocol (TCP/UDP/ICMP) individually. The output of jool -b --count would also need to have the same granularity, so that all the reported numbers could be compared in a meaningful fashion.

Alternatively: If jool -4 --count would report both total number of transport identifiers and the number of used transport identifiers peer pool+protocol I'd get away with executing the jool binary once per monitoring cycle. That'd be nice.

[ydahhrk]

That is, for each individual IPv4 pool (as distinguished by --mark) report the number of transport identifiers for each protocol (TCP/UDP/ICMP) individually.

OK

The output of jool -b --count would also need to have the same granularity, so that all the reported numbers could be compared in a meaningful fashion.

BIB doesn't care about marks, so this'd be hard right now.

Alternatively: If jool -4 --count would report both total number of transport identifiers and the number of used transport identifiers peer pool+protocol I'd get away with executing the jool binary once per monitoring cycle. That'd be nice.

Well, pool4 now only worries about address space available for NAT64ing in general. Actual utilization is only BIB's concern. So if we want to join these two kinds of outputs, I wager it'd be really counter-intuitive/misleading if it's done under the pool4 "brand". I'd be more inclined to create a new configuration "mode" for this.

I'm fantasizing ways to do this really efficiently, but they involve more internal refactors. I didn't think about utilization analysis when I designed the databases. (--count was born as a quick 'n dirty debugging tool and only stayed because of inertia.) The most straightforward way to do this now would be to just dump the output of jool -4 --csv into a hash table, and then foreach the output of jool -b --csv, increasing counters. It wouldn't even need to be done by jool.

What I'm getting at is, if you'd rather have this quickly than clean, I could probably pull it off in a Java application comfortably in like 30 minutes (it's my native language) and have it uploaded to a separate project. Would it be awkward to install the Java Virtual Machine in whichever node you want to run the analysis in?

(Update: Actually, this way of doing it would probably be both quicker and cleaner, since it would prevent Jool's inner code from getting more complicated.)

[toreanderson]

Don't worry, I'll be able to do what you propose in my native Perl as well. :-) It's not as simple as my old way - essentially multiplying the output from jool -4 --count with 2^16 to determine the total number of transport identifiers for each protocol, and comparing that with the three outputs from jool -b --count - but it's certainly doable.

So don't stress about doing any internal refactoring or anything just for this, but if you're touching that code anyway at some point in the future it would be convenient to also ensure the utilisation information becomes easily available - for example, if you use Jool as a CGN then you would like some early warning if your pool4 is about to run out of space, because once it does stuff stops working, users gets angry, and so on. So it's preferable to get the chance to add some more addresses in proactive fashion.

[ydahhrk]

Ok, might want to request a pull of the script or upload it somewhere, since it might help somebody else too.

(In the meantime.)

(BTW: now that I think about it, the hash table approach doesn't work because pool4 entries are allowed to intersect - ie. a BIB entry can belong to more than one pool4 entry (wearing different marks).)

(In fact, a BIB entry can belong to zero pool4 entries if it has been orphaned.)

[toreanderson]

Ok, might want to request a pull of the script or upload it somewhere, since it might help somebody else too.

I should indeed. It'll have to wait until the new year though, as I'm in very much holiday mode already...

[ydahhrk]

Done: https://github.com/NICMx/pool4-usage-analyzer