search

renaissance-benchmarks / renaissance

The Renaissance Benchmark Suite
https://renaissance.dev
GNU General Public License v3.0
313 stars 60 forks source link

db-shootout does not work on linux ppc64le / s390x #153

Closed MBaesken closed 2 years ago

MBaesken commented 5 years ago

It seems the db-shootout benchmark does not work on linux ppc64le / s390x . It contains a native lib (leveldbjni64-1.8 ) that is not available on all CPU/HW platforms where OpenJDK is supported. Error message (example is from linux ppc64le) java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.8 in java.library.path, no leveldbjni-1.8 in java.library.path, no leveldbjni in java.library.path, /tmp/libleveldbjni-64-1-8490766625802501468.8: /tmp/libleveldbjni-64-1-8490766625802501468.8: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64-bit .so on a Power PC 64 LE-bit platform)]

Probably the db-shootout benchmark should be run only on the platforms that are supposed to work (means: has the native library available).

MBaesken commented 5 years ago

btw. the reported issues has been seen with both JDK8 and JDK11.

farquet commented 5 years ago

Thanks Matthias for the report. The leveldbjni-all dependency is "supposed" to support all platforms, unfortunately, it doesn't seem to be the case as one can see by unzipping the jar at http://central.maven.org/maven2/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar

I see different options here :

  1. Add the missing native libs to a fork of leveldbjni-all and push it to maven since the owners don't seem willing to do it : https://github.com/fusesource/leveldbjni/issues/85
  2. Retire the benchmark since it would be convenient not to have benchmarks relying on native libs for compatibility reasons. That would require a good replacement benchmark though.
  3. Mark supported platforms for each benchmark and skip them when running on an unsupported platform.
  4. Remove LevelDB from that benchmark since the benchmark consists of four different DBs tested. That probably wouldn't hurt so much to go down to three. https://github.com/renaissance-benchmarks/renaissance/blob/master/benchmarks/database/src/main/scala/org/renaissance/database/DbShootout.scala#L69-L91

Any opinion welcome @lbulej, @vhotspur, @axel22 & co.

I personally think that option 4 may be a good tradeoff. It's a simple change that would solve that particular issue. Moreover, the benchmark would probably not behave very differently. That would need to be quantify though.

axel22 commented 5 years ago

I also think that option 4 is preferable for now.

lbulej commented 5 years ago

I also think that option 4 is preferable for now.

At the moment, adding support for 3 would be quite far in the priority chain of design changes, so I'd support 4 as well.

axel22 commented 5 years ago

Wouldn't the list of supported platforms for each benchmark just be a property associated with a benchmark?

farquet commented 5 years ago

Yes, that would be just a property, but if we can slightly change the problematic benchmark(s) to work on most platforms, that’s one less dimension to care about.

I will tackle the implementation of 4 by measuring the performance difference on several VMs.

farquet commented 5 years ago

@MBaesken, the benchmark has been fixed to remove the native dependency. Please give it a try and reopen the issue if there are remaining problems.

You can build a jar from master, or wait for the 0.10.0 release which will come very soon.

MBaesken commented 5 years ago

When testing the latest renaissance-mit-0.10.0.jar db-shootout I still run into issues with native libs that are not found .

AIX :

/rs6000_64/nightly/output-jdk8/images/j2sdk-image/bin/java -jar /benchmarks/renaissance/renaissance-mit-0.10.0.jar db-shootout SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Error during tear-down: null java.lang.NullPointerException at org.lmdbjava.bench.Chronicle$CommonChronicleMap.teardown(Chronicle.java:131) at org.lmdbjava.bench.Chronicle$Reader.teardown(Chronicle.java:210) at org.renaissance.database.DbShootout.tearDownAfterAll(DbShootout.scala:85) at org.renaissance.RenaissanceBenchmark.runBenchmark(RenaissanceBenchmark.java:102) at org.renaissance.RenaissanceSuite$.$anonfun$main$2(renaissance-suite.scala:298) at org.renaissance.RenaissanceSuite$.$anonfun$main$2$adapted(renaissance-suite.scala:296) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at org.renaissance.RenaissanceSuite$.main(renaissance-suite.scala:296) at org.renaissance.RenaissanceSuite.main(renaissance-suite.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.renaissance.Launcher.main(Launcher.java:21) Exception occurred in org.renaissance.database.DbShootout@2f666ebb: Native library (com/sun/jna/aix-ppc64/libjnidispatch.so) not found in resource path ... ....... at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:866) at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) at com.sun.jna.Native.(Native.java:140) at com.sun.jna.NativeLibrary.(NativeLibrary.java:84) at net.openhft.chronicle.hash.impl.util.jna.PosixMsync.(PosixMsync.java:28) at net.openhft.chronicle.hash.impl.VanillaChronicleHash.msync(VanillaChronicleHash.java:845) at net.openhft.chronicle.hash.impl.VanillaChronicleHash.msync(VanillaChronicleHash.java:831) at net.openhft.chronicle.map.ChronicleMapBuilder.commitChronicleMapReady(ChronicleMapBuilder.java:413) at net.openhft.chronicle.map.ChronicleMapBuilder.createWithNewFile(ChronicleMapBuilder.java:1749) at net.openhft.chronicle.map.ChronicleMapBuilder.createWithFile(ChronicleMapBuilder.java:1652) at net.openhft.chronicle.map.ChronicleMapBuilder.createPersistedTo(ChronicleMapBuilder.java:1552) at org.lmdbjava.bench.Chronicle$CommonChronicleMap.setup(Chronicle.java:122) at org.lmdbjava.bench.Chronicle$Reader.setup(Chronicle.java:204) at org.renaissance.database.DbShootout.setUpBeforeAll(DbShootout.scala:73) at org.renaissance.RenaissanceBenchmark.runBenchmark(RenaissanceBenchmark.java:79) at org.renaissance.RenaissanceSuite$.$anonfun$main$2(renaissance-suite.scala:298) at org.renaissance.RenaissanceSuite$.$anonfun$main$2$adapted(renaissance-suite.scala:296) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at org.renaissance.RenaissanceSuite$.main(renaissance-suite.scala:296) at org.renaissance.RenaissanceSuite.main(renaissance-suite.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.renaissance.Launcher.main(Launcher.java:21)

linuxs390x (OpenJDK jdk11, I had no jdk8 at hand) :

/linuxs390x/nightly/output-jdk11-test/images/jdk/bin/java -jar /benchmarks/renaissance/renaissance-mit-0.10.0.jar db-shootout SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Error during tear-down: null java.lang.NullPointerException at org.lmdbjava.bench.Chronicle$CommonChronicleMap.teardown(Chronicle.java:131) at org.lmdbjava.bench.Chronicle$Reader.teardown(Chronicle.java:210) at org.renaissance.database.DbShootout.tearDownAfterAll(DbShootout.scala:85) at org.renaissance.RenaissanceBenchmark.runBenchmark(RenaissanceBenchmark.java:102) at org.renaissance.RenaissanceSuite$.$anonfun$main$2(renaissance-suite.scala:298) at org.renaissance.RenaissanceSuite$.$anonfun$main$2$adapted(renaissance-suite.scala:296) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at org.renaissance.RenaissanceSuite$.main(renaissance-suite.scala:296) at org.renaissance.RenaissanceSuite.main(renaissance-suite.scala) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.renaissance.Launcher.main(Launcher.java:21) Exception occurred in org.renaissance.database.DbShootout@29a60c27: Native library (com/sun/jna/linux-s390x/libjnidispatch.so) not found ...... at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:866) at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) at com.sun.jna.Native.(Native.java:140) at net.openhft.chronicle.hash.impl.util.jna.PosixFallocate.(PosixFallocate.java:18) at net.openhft.chronicle.hash.impl.VanillaChronicleHash.map(VanillaChronicleHash.java:977) at net.openhft.chronicle.hash.impl.VanillaChronicleHash.createMappedStoreAndSegments(VanillaChronicleHash.java:485) at net.openhft.chronicle.map.ChronicleMapBuilder.createWithNewFile(ChronicleMapBuilder.java:1748) at net.openhft.chronicle.map.ChronicleMapBuilder.createWithFile(ChronicleMapBuilder.java:1652) at net.openhft.chronicle.map.ChronicleMapBuilder.createPersistedTo(ChronicleMapBuilder.java:1552) at org.lmdbjava.bench.Chronicle$CommonChronicleMap.setup(Chronicle.java:122) at org.lmdbjava.bench.Chronicle$Reader.setup(Chronicle.java:204) at org.renaissance.database.DbShootout.setUpBeforeAll(DbShootout.scala:73) at org.renaissance.RenaissanceBenchmark.runBenchmark(RenaissanceBenchmark.java:79) at org.renaissance.RenaissanceSuite$.$anonfun$main$2(renaissance-suite.scala:298) at org.renaissance.RenaissanceSuite$.$anonfun$main$2$adapted(renaissance-suite.scala:296) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at org.renaissance.RenaissanceSuite$.main(renaissance-suite.scala:296) at org.renaissance.RenaissanceSuite.main(renaissance-suite.scala) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.renaissance.Launcher.main(Launcher.java:21)

So it looks like there are native dependencies remaining.

axel22 commented 5 years ago

I think we should maybe think about including the Apache Ignite database in this benchmark instead.

farquet commented 5 years ago

Thanks @MBaesken for the testing. This is where the native library is loaded in Chronicle code : https://github.com/OpenHFT/Chronicle-Map/blob/master/src/main/java/net/openhft/chronicle/hash/impl/util/jna/PosixMsync.java#L28

I'll investigate how much it take to replace Chronicle by Ignite.

lbulej commented 3 years ago

Thanks @MBaesken for the testing. This is where the native library is loaded in Chronicle code : https://github.com/OpenHFT/Chronicle-Map/blob/master/src/main/java/net/openhft/chronicle/hash/impl/util/jna/PosixMsync.java#L28

I'll investigate how much it take to replace Chronicle by Ignite.

Or we can just update the JNA library used by Chronicle to call msync. JNA 5.9.0 seems to have support for ppc64le/s390x, so it might help. I did the update in an attempt to fix one of the issues in #296 (which is common to this one) so it might be relevant to this issue as well. See my comment in #296 for link to a custom build of Renaissance containing only db-shootout with the updated JNA library.

MBaesken commented 3 years ago

Linux ppc64le worked nicely when testing with renaissance-gpl-0.13.0-4-g2755c79.jar (both with our JDK8 and JDK11).
AIX worked too when testing with JDK8 (had no JDK11 at hand). Linux s390x worked with JDK11; JDK8 failed with the exception 

Benchmark 'db-shootout' failed with exception:
java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/linux-zarch_64/libjnidispatch.so) not found in resource path

While os.arch is s390x in JDK11, it was still zArch_64 in 8 . This might lead to the error.

lbulej commented 3 years ago

@MBaesken That's excellent news! It seems that mapping from zArch_64 to s390x should be easy enough to handle in JNA, so I guess we can delay merging #322 until the fix makes it to JNA. Are you going to open an issue with the JNA people?

MBaesken commented 3 years ago

I opened https://github.com/java-native-access/jna/issues/1388 at JNA regarding the os.arch mapping; let's see what happens.

farquet commented 3 years ago

Can this be closed now?

MBaesken commented 3 years ago

Hi, my change for issue https://github.com/java-native-access/jna/issues/1388 went into JNA . There is already a JNA 5.10 containing it (Map the arch zarch_64 as reported by SAPJVM8) : https://github.com/java-native-access/jna/blob/master/CHANGES.md#release-5100 did you switch to this new version already , if so then the issue can be closed.

farquet commented 3 years ago

@lbulej's PR bumps to the new version: https://github.com/renaissance-benchmarks/renaissance/pull/322

MBaesken commented 3 years ago

Sounds great - any idea when the new release with this change approx. will be released ? (and yes I think we can close this issue)

lbulej commented 2 years ago

Sounds great - any idea when the new release with this change approx. will be released ? (and yes I think we can close this issue)

The plan was to get a maintenance update out by the beginning of December but we wanted to make it easier to run the benchmarks on platforms that cannot do multiple class loaders. If things go well now, the first half of January 2022 looks like a reasonable target.

lbulej commented 2 years ago

It took a little longer, but the updated JNA that fixes the problem is now part of the 0.14.0 release. Thank you for helping with this issue!