BuaBook / kdb-common

kdb+ Core Libraries and Utilities
Apache License 2.0
51 stars 16 forks source link

log: Add support for pattern-based logging and call line tracing #54

Closed jasraj closed 3 years ago

jasraj commented 3 years ago

Performance Testing

Equivalent Logging

Comparing the new pattern-based logging (increased flexibility) vs the old logger (fixed):

/ model name      : Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz

/ Setup
q) dict:`a`b`c!1 2 3;

/ For new pattern-based logging (matching original logger)
q) .log.cfg.format:"%d %t %l %p %u %h %m"
q) .log.setLogger[]

q) .log.info "A dictionary: ",.Q.s1 dict;                                   / Original
q) .log.info ("A dictionary: {}"; dict);                                    / Parameterised

Results:

Formatter Log Format Iterations ts
Fixed Original 1000 20 4195376
Pattern Original 1000 22 4195376
Fixed Parameterised 1000 21 4195264
Pattern Parameterised 1000 24 4195264

The pattern-based logging has ~10% degradation in performance when logging with the same elements as the original logging.

Call Trace Logging

/ model name      : Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz

/ Setup
q) .test.log:{ .log.info "Logging test" };

/ Original logger pattern
q) .log.cfg.format:"%d %t %l %p %u %h %m"
q) .log.setLogger[]

/ Logger pattern with call trace logging
q) .log.cfg.format:"%d %t %l %p %u %h %T %m"
q) .log.setLogger[]

q) \ts do[1000; .test.log[]; ]

Results:

Call Tracing Iterations ts
NO 1000 12 4195088
YES 1000 28 4195088

Including call-trace logging in the formatting has ~130% degradation in performance