search

dice-group / IGUANA

IGUANA is a benchmark execution framework for RDF triple stores and quad stores
http://iguana-benchmark.eu
GNU Affero General Public License v3.0
24 stars 15 forks source link

Improve benchmark execution performance #244

Closed nck-mlcnv closed 3 weeks ago

nck-mlcnv commented 6 months ago

The measurement of the AvgQPS metric, which is sensitive to short query execution times, is limited by how fast Iguana can handle the responses from queries. Therefore improving the performance of Iguana could lead to more reliable and accurate results. Potential performance improvements:

HTTP Client from scratch

This one has already been tested with an improvised implementation here. It utilizes the java.net library for the socket communication and is only capable of sending GET requests and receiving simple responses, but improvements were still noticeable with this implementation.

Ahead-of-Time compilation

Use GraalVM for the ahead-of-time compilation of Iguana. This should improve the performance by removing the need of the JIT, which optimizes the compiled bytecode during query executions. This should also heavily improve the warmup-phase of Iguana. The downsides are, that Log4j2 is not supported with GraalVM.

Delegate benchmark execution to external program

There is also the option to delegate the code for the benchmark executions to an external, natively compiled library that has been written in Rust and to call it through the JNI (or similar).

bigerl commented 3 weeks ago

GraalVM is supported now. Closing.