fullstorydev / solr-bench

Solr benchmarking and load testing harness
Apache License 2.0
17 stars 10 forks source link

Benchmarking Indexing formats: Javabin, JSON & CBOR #70

Closed chatman closed 1 year ago

chatman commented 1 year ago

For https://issues.apache.org/jira/browse/SOLR-16812, we needed a way to test performance of indexing formats like CBOR versus JSON and Javabin.

Introducing a "prepare-binary-format" initializing phase before actual indexing, where the binary files for each batch is generated. During indexing phase, the binary files are uploaded instead of uploading the docs one by one. Since batching during indexing is deterministic, the batches initialized line up exactly with the batches indexed.