kytos.core.log can contribute to slowness on hot paths

[viniarck]

This is the same as what was originally reported o flow_manager issue 77, however, I'm mapping it here on kytos core since the solution will need to be implemented here to provide a queued and/or async logger

[viniarck]

It turns out that this function _detect_napp_id that used to dynamically detect a NApp id is adding significant latency, contributing to a significant part

An alternative is not to use log from kytos.core and instead explicitly set a new one for a NApp, for example on of_core it would be:

log = logging.getLogger("kytos.napps.kytos/of_core")

[viniarck]

I've also run more benchmark tests comparing the current log form kytos.core and getting the logger directly logging.getLogger("kytos.napps.kytos/sample_ui"), I've used this sample_ui napp. I've tested it with 10 requests/sec:

• current log from kytos.core that the bottleneck with _detect_napp_id has been identified.

• logging.getLogger("kytos.napps.kytos/sample_ui"), the result were great, the log.debug statement itself most of the time considering its own 95 percentile is 200us, however there can be some outliers that can go over 1ms, which isn't as unstable:

As we've briefly discussed, QueueHandler looks promising we could also use a janus Queue, passing a sync_q since it'll use a put_nowait, as long as the maxsize is 0 it should be ok, and we could use it with QueueListener, and then we could stop the QueueListener whenever the core is shutdown (same way as the ThreadPools and tasks are managed). Using these two should be simpler than rolling out and maintaining another long-running Thread or new asyncio.Task with new async-friendly logger library. These new log instance should also be provided in the core, but we also want to remove the _detect_napp_id, maybe, we could parametrize a function that passes a string of the NApp id, and then the function prefixes it with kytos.napps

[viniarck]

Closing this since PRs https://github.com/kytos-ng/kytos/issues/250 and https://github.com/kytos-ng/kytos/pull/247 have enhanced this part, so now, log entries aren't as slow as they used to be. When in doubt, if you'll introduce in a hot path, it's also not expect to delay much, but when in doubt, measure/instrument with APM and see if you can use it.