search

BitFunnel / mg4j-workbench

Java tools for evaluating BitFunnel performance compared to an mg4j baseline.
GNU Lesser General Public License v3.0
1 stars 2 forks source link

Mitigate performance impact of slf4j logging #1

Closed MikeHopcroft closed 7 years ago

MikeHopcroft commented 7 years ago

Currently the single-query version of QueryEngine.Process() (starts at line 256 of QueryEngine.java) logs the fact that it is processing a query. This happens on line 257. 

LOGGER.debug( "Processing query \"" + queries + "\", offset=" + offset + ", length="+ length );

This call to the slf4j logging will cause the parameter string to be serialized, which is likely to be expensive. See this note on slf4j logging performance for more information.

Note that using the nop logger (slf4j-nop-1.8.0-alpha1.jar) will not solve this performance problem because the parameter string will still be created.

One possible solution is to use the query-array version of QueryEngine.Process() (starts on line 298). This version logs once per call, instead of once per query. The challenge with this version is that storing all of the matches for all of the queries might lead to GC-related performance problems.

Another possible solution is to incorporate the mg4j source code directly into our repository (instead of allowing maven to reference the .JAR files). With this strategy, we could comment out the logging.

MikeHopcroft commented 7 years ago

I forked QueryEngine to ExperimentalQueryEngine and then erased the calls to the logger.