One of the optimizations done in the tbm-common-parser-lib 2.0 release was to add a new feature to ITokenizer which allowed it to reuse the same IToken instance over and over again as the return value from nextToken() instead of creating a new instance.
The performance (CPU time and especially memory) benefits were significant for long-running parser operations and the perfect use-case is one where the IToken is not being exposed to the caller where they might assume it was a non-volatile object.
LogParser is exactly this use case, so this functionality needs to be enabled.
ghost
commented
13 years ago
One of the optimizations done in the tbm-common-parser-lib 2.0 release was to add a new feature to ITokenizer which allowed it to reuse the same IToken instance over and over again as the return value from nextToken() instead of creating a new instance.
The performance (CPU time and especially memory) benefits were significant for long-running parser operations and the perfect use-case is one where the IToken is not being exposed to the caller where they might assume it was a non-volatile object.
LogParser is exactly this use case, so this functionality needs to be enabled.
This would justify a new release.