search

fullstorydev / solr-bench

Solr benchmarking and load testing harness
Apache License 2.0
17 stars 10 forks source link

Support of pluggable task class #88

Closed patsonluk closed 9 months ago

patsonluk commented 9 months ago

Description

solr-bench supports several tasks (indexing/querying etc) and those are embedded in the existing BenchmarksMain and StressMain. It seems that adding more tasks will likely increase the complexity of those code hence we are adding capability of pluggable task class.

Other users can implement the new Task interface or the AbstractTask class, the implementation can be placed inside the new org.apache.solr.benchmarks.task package or be supplied via other jar and loaded w/o modifying the solr-bench code. The test config can then reference such class by the class name.

The full description of usage is documented in https://github.com/fullstorydev/solr-bench/tree/patsonluk/custom-task-class/src/main/java/org/apache/solr/benchmarks/task#readme

Solution

  1. Added Task interface and AbstractTask class. For sub-class of AbstractTask we assume that it shares the same config as BaseBenchmark, which means it also support basic params such as rpm, duration-secs etc. The task defined by the implementation is submitted to the ControlledExecutor which has more consistent and centralized control on task execution
  2. Added a subclass SegmentMonitoringTask which monitors total segment count/segment doc size etc, currently this only support reporting metrics to the Prometheus listener
  3. Re-factored the logic of Prometheus listener/server. It should now be able to handle more generic usage for reporting metrics via Gauge and Histogram
  4. Replaced ex.printStackTrace() with logger call. Also added waitForSubmittedTasks method which should interrupt all submitted task and print the exception if any of the submitted tasks failed with unhandled exception. The behavior before was that all tasks would still keep running on unexpected exception and such exception would only be surfaced once all the tests are done
nginthfs commented 9 months ago

Would it be hard to separate the exception handling code into a separate PR for easier review?

patsonluk commented 9 months ago

Would it be hard to separate the exception handling code into a separate PR for easier review?

They are in these 4 commits :)

https://github.com/fullstorydev/solr-bench/compare/8a308df883b5853603580edb199229756734f883...f0eae064e55163e8f4e167f6a4261ff800bfd5f3

patsonluk commented 9 months ago

@nginthfs Thank you for the detailed reviews, I will merge this in.

We can adjust this by small commits if there are any followups!