Java benchmark fails to run

van-smith commented 13 years ago

The Java benchmark launches, but fails to run. The GUI remains up but no progress is made. The harness remains hung and does not terminate the test despite the "will timeout in 12 minutes" message in the HUD.

This has happened with the latest code pull from this morning and it is repeatable.

ghost commented 13 years ago

Issue originated after a conversion of C++ projects from V90 toolset+multi-threaded DLLs to V100 toolset+static libraries. Root cause of issue could not be determined. A reversion of source code to a prior "known good" state, and then a subsequent re-introduction of each change made thereafter up to and including the point of observed failure revealed a clean path of both a proper source code design and generated executable code in both the V90 and V100 toolsets. While it remains unknown what caused the error, it has been corrected and is now functioning properly.

van-smith commented 13 years ago

Fix confirmed by Kathy

van-smith commented 12 years ago

This issue has remanifested after I made unrelated changes to the Java code to correct test behaviors for STREAM and Strings. I will not push the new code until I can find a workaround. It is discouraging that Rick never found the solution to the original failure.

van-smith commented 12 years ago

There were many bugs in the Java benchmark. Here are a few that I have fixed so far:

Rick H. never identified a fix for bug #96. However, the fixed benchmark.jar was not built from code in the repository: building the JAR from the repository would reintroduce bug #96. I determined that the bug came from instantiating the tests within the benchmark constructor. I moved the test instantiation into specific routines that launched the tests.
The Strings test needed to be rewritten because it was not designed as specified.
The STREAM data sets were too small and had the potential to be influenced by large caches.
STREAM was over-represented in the Java test results. STREAM now returns a single result.
The Java tests were being called in a way that did not allow optimization flags or heap size settings.
Core affinity was not being calculated properly.
Core affinity was not being effectively applied.

I have also seen that Java benchmark scores are not properly corresponding to reported times. I will investigate the issue after I update bug #117.

van-smith / OPBM

Java benchmark fails to run #96