cmd/audit: introduce new stats option

[p0lyn0mial]

the new option calculates and displays a summary of statistical metrics (max, min, median, 90th percentile) for the latency trackers (measure latency incurred in components within the apiserver) stored in the audit logs for http requests.

usage:

./kubectl-dev_tool audit -f /audit.log -o stats

Summary of the requests latencies (analysed requests: 37392):
======================================================================
apiserver.latency.k8s.io/decode-response-object   : max=241.274µs min=60.191µs median=196.574µs 90th=227.009µs
apiserver.latency.k8s.io/etcd                     : max=1m0.003963671s min=28.052µs median=14.998918564s 90th=59.999428921s
apiserver.latency.k8s.io/mutating-webhook         : max=0s min=0s median=0s 90th=0s
apiserver.latency.k8s.io/response-write           : max=107.110279ms min=151ns median=591ns 90th=1.212µs
apiserver.latency.k8s.io/serialize-response-object: max=19.255044ms min=15.179µs median=2.624864ms 90th=4.179904ms
apiserver.latency.k8s.io/total                    : max=2m35.352837698s min=501.720592ms median=15.002956031s 90th=1m0.00091996s
apiserver.latency.k8s.io/transform-response-object: max=731ns min=40ns median=151ns 90th=330ns
apiserver.latency.k8s.io/validating-webhook       : max=3.077167ms min=67.034µs median=2.198373ms 90th=2.873546ms

[p0lyn0mial]

/assign @tkashem

[p0lyn0mial]

can it work alongside audit filter? for example, i want to see the latency summary for requests originating from a certain user, or coming from a particular namespace?

yes, you can apply filters as usual. the filtered events are then passed to the new command.

I would like to see the number of events processes, and their time span. The audit filter does that I believe

good idea, I will add more info to the summary.

[p0lyn0mial]

@tkashem thanks for the review. I've applied your suggestions. PTAL.

[p0lyn0mial]

@tkashem updated the PR, PTAL.

[tkashem]

/lgtm

[p0lyn0mial]

/hold cancel