Closed PanagiotisDrakatos closed 5 months ago
@gunnarmorling please check again i believe this is my last attempt to improve performance after that my capabilities are limited
Runs out of heap space for the tests on 32 cores:
Caused by: java.lang.OutOfMemoryError: Java heap space
at dev.morling.onebrc.CalculateAverage_PanagiotisDrakatos$MeasurementRepository.<init>(CalculateAverage_PanagiotisDrakatos.java:274)
at dev.morling.onebrc.CalculateAverage_PanagiotisDrakatos.MappingByteBufferToData(CalculateAverage_PanagiotisDrakatos.java:164)
at dev.morling.onebrc.CalculateAverage_PanagiotisDrakatos$$Lambda/0x00007f6b43001860.apply(Unknown Source)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:960)
at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:934)
at java.base/java.util.stream.AbstractTask.compute(AbstractTask.java:327)
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
@gunnarmorling i fixed VM options arguments and increased the xmx. Probably it would not failed now
Looking good now: 00:05.559. Passing for 10K keys, too.
Check List:
[ ] You have run
./mvnw verify
and the project builds successfully[ ] Tests pass (
./test.sh <username>
shows no differences between expected and actual outputs)[ ] All formatting changes by the build are committed
[ ] Your launch script is named
calculate_average_<username>.sh
(make sure to match casing of your GH user name) and is executable[ ] Output matches that of
calculate_average_baseline.sh
[ ] For new entries, or after substantial changes: When implementing custom hash structures, please point to where you deal with hash collisions (line number)
Execution time:
Execution time of reference implementation: