Closed patsonluk closed 1 year ago
I haven't tested these changes, but nothing jumps out as obviously wrong. Thanks for tackling this.
Thank you so much @chatman! This is one sizeable refactoring so take your time!
I don't have time yet to write any unit test cases ๐ but I did some simple testing on my end (mostly single threaded tho):
I have also started using that for our use case - having light indexing and heavy querying concurrently with fixed duration for both.
Would be great if you can test on your use cases as well! ๐ช๐ผ
Sure, I'll update you by tomorrow on how the testing goes. THanks!
@chatman thanks for approving it! Im wondering if you have a chance to run through the test cases on ur end too? ๐
Im wondering if you have a chance to run through the test cases on ur end too?
I'll test them today. Thank your for the patience :-)
@patsonluk , I ran them on a few suites yesterday and didn't notice any problems.
I don't have time yet to write any unit test cases
I've created an issue here, we can revisit later. https://github.com/fullstorydev/solr-bench/issues/53
Description
A refactoring effort to address:
duration-sec
is defined forIndexBenchmark
but it currently has no effectControlledExecutor
which controls duration/rate for querying testControlledExecutor
has certain discrepancy for the rpm configured vs the actual rpm achieved especially in high throughput scenariosSolution
IndexBatchSupplier
which "supply" tasks (Callable
) which each callable is a index batch writing operation, refactored existing indexing logic to use such supplier and pass that toControlledExecutor
. SuchIndexBatchSupplier
ensures better separation of concerns of the 2 major tasks:DocReader
/FileDocReader
that respectmaxDocs
)ControlledExecutor
such that the controls (duration/rpm) are more well defined and precisely respected.This is a pretty big refactoring, but I'm hoping this could simplify/solidify the query/index design and code implementation. Could really use some code review and thoughts! @chatman Many thanks! ๐๐ผ