search

Iyengar111 / NanoLog

Low Latency C++11 Logging Library
727 stars 186 forks source link

Please show also worst case latencies #1

Closed KjellKod closed 8 years ago

KjellKod commented 8 years ago

Sweet library! Nice work

I found the bench mark lacking in a couple of aspects

  1. It doesn't show how the logger performce under congestion. I.e when the buffer gets full
  2. Could you please also add worst case latencies and not only average? For time critical operations one cares less about the average as how the slowest log call does (especially when the ring buffer gets full).

Thanks Kjell

Iyengar111 commented 8 years ago

Hi Kjell,

Thanks for having a look.

In terms of the design, when the ring buffer is full, the producer just overwrites the log line in that slot [i.e. does not wait for consumer thread to pop the item]. In summary - the old log line will be lost when buffer is full.

In the event this occurs, its a sign the ring buffer size passed on initialization was chosen incorrectly relative to the logging frequency of the application. The end-user simply has to choose a more appropriate ring buffer size.

That's why I feel overwriting the old log line in the extreme case is the correct design choice. The added advantage is that in theory the latency is independent of the state of the ring buffer [because full or not, the producer always writes the log line to slot and does not care if there was a previous log line in the slot].

In terms of measuring worst case latency there are complications -

  1. NanoLog takes around 100nanoseconds for a line.
  2. Timestamps from chrono / gettimeofday / clock gettime take ~40nanos to complete.
uint64_t begin = timestamp_now();
LOG_INFO << "Benchmark this";
uint64_t end = timestamp_now();

The error in measuring NanoLog latency like this is too high relative to the time NanoLog takes.

That's why in the benchmark, I measured the time taken to log 1000 lines (which would be in 100s of microseconds ) which can be measured accurately by chrono timestamps.

I do not know of any accurate way to measure individual log line latency...

Karthik

KjellKod commented 8 years ago

Cool. Thanks for explaining this.

I see why the "fire and forget" policy to logs is appealing. (I.e overwrite)