arriven / db1000n

MIT License
1.18k stars 208 forks source link

Combine metrics summary into one object #518

Closed roman-kruglov closed 2 years ago

roman-kruglov commented 2 years ago

so the whole report goes as one log entry with a JSON object for each target

Description

This might be helpful for logging and assessing to have the whole summary report as one entry. Though arguably it makes it even less readable in terminal's output, and I tried to improve that with pr #517. I tried to make zap pretty print this huge JSON report object (for the case when -log-format simple from #517), but it was way too complicated for such a simple feature. Though it might still be possible.

What do you guys think? Could this be helpful?

Type of change

Potentially somebody could rely on the current summary format, but with so many swift changes and the recent move to zap - I doubt it.

How Has This Been Tested?

By running and looking at the output in a terminal.

Test Configuration

Logs

Output sample (real values crossed out, truncated for shorter text)

{"http://sample": {"requests_attempted": 7, "requests_sent": 0, "responses_received": 0, "bytes_sent": 0}, "http://XX.XX.XX.XX": {"requests_attempted": 7, "requests_sent": 0, "responses_received": 0, "bytes_sent": 0}, "http://sample": {"requests_attempted": 7, "requests_sent": 0, "responses_received": 0, "bytes_sent": 0}, "total": {"requests_attempted": 185664, "requests_sent": 183986, "responses_received": 138, "bytes_sent": 1862621}}

the full log entry with the current default format would look like this:

{"level":"info","ts":1650809773.640194,"caller":"metrics/metrics.go:183","msg":"stats","http://sample":{"requests_attempted":3,"requests_sent":3,"responses_received":3,"bytes_sent":624},"http://sample":{"requests_attempted":1,"requests_sent":1,"responses_received":1,"bytes_sent":191},"http://sample":{"requests_attempted":1,"requests_sent":1,"responses_received":1,"bytes_sent":197},"tcp://XX.XXX.X.XX:XX":{"requests_attempted":91924,"requests_sent":91924,"responses_received":0,"bytes_sent":919240},"total":{"requests_attempted":184496,"requests_sent":183902,"responses_received":54,"bytes_sent":1848272}}
roman-kruglov commented 2 years ago

I have several more ideas regarding this and simple output too, I'll have more improvements

arriven commented 2 years ago

I've added a change that groups up all the targets into an object to simplify parsing (you can safely iterate on all keys in targets json object instead of having to filter root keys), please be aware

roman-kruglov commented 2 years ago

I've added a change that groups up all the targets into an object to simplify parsing (you can safely iterate on all keys in targets json object instead of having to filter root keys), please be aware

yep, was thinking about combining targets for this same purpose too