search

jvz / future-promise

A port of the Future/Promise API from Scala for Java.
1 stars 0 forks source link

Add JMH benchmarks comparing other futures #1

Open jvz opened 5 years ago

jvz commented 5 years ago

Some JMH benchmarks should be written and executed comparing the performance of this implementation of Future to CompletableFuture from Java 8, Future from other ExecutorService implementations, and perhaps some other implementations such as ListenableFuture from Guava.

See upstream Scala tests for comparison: https://github.com/scala/scala/blob/2.13.x/test/benchmarks/src/main/scala/scala/concurrent/FutureBenchmark.scala

jvz commented 5 years ago

Potential alternative might be to fork that benchmark, though then we'd be benchmarking the library when used in Scala, not Java.

jvz commented 5 years ago

Another class to compare DefaultPromise to would be https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/FutureTask.html

jvz commented 5 years ago

And there's a rather simple Future/Promise implementation in vavr that can be compared (fairly similar API as it's based on Scala's API, too).

jvz commented 5 years ago

Initial findings for basic benchmark porting from Scala benchmarks (plus fixes):

Benchmark                 (pool)  (recursion)  (threads)   Mode  Cnt    Score   Error   Units
MapFutureBenchmark.post      fjp         1024          1  thrpt   10   10.671 ± 0.113  ops/ms
MapFutureBenchmark.post      fix         1024          1  thrpt   10   11.021 ± 0.077  ops/ms
MapFutureBenchmark.post      fie         1024          1  thrpt   10   13.557 ± 0.073  ops/ms
MapFutureBenchmark.post      gbl         1024          1  thrpt   10   10.594 ± 0.160  ops/ms
MapFutureBenchmark.pre       fjp         1024          1  thrpt   10   18.533 ± 0.470  ops/ms
MapFutureBenchmark.pre       fix         1024          1  thrpt   10   15.486 ± 0.160  ops/ms
MapFutureBenchmark.pre       fie         1024          1  thrpt   10   22.835 ± 0.126  ops/ms
MapFutureBenchmark.pre       gbl         1024          1  thrpt   10   18.413 ± 0.544  ops/ms
NoopFutureBenchmark.post     fjp         1024          1  thrpt   10  336.289 ± 4.505  ops/ms
NoopFutureBenchmark.post     fix         1024          1  thrpt   10  337.467 ± 2.082  ops/ms
NoopFutureBenchmark.post     fie         1024          1  thrpt   10  337.807 ± 1.807  ops/ms
NoopFutureBenchmark.post     gbl         1024          1  thrpt   10  337.586 ± 1.882  ops/ms
NoopFutureBenchmark.pre      fjp         1024          1  thrpt   10  397.252 ± 2.725  ops/ms
NoopFutureBenchmark.pre      fix         1024          1  thrpt   10  397.322 ± 2.162  ops/ms
NoopFutureBenchmark.pre      fie         1024          1  thrpt   10  391.944 ± 8.108  ops/ms
NoopFutureBenchmark.pre      gbl         1024          1  thrpt   10  397.962 ± 2.541  ops/ms