search

van-smith / OPBM

Open Productivity Benchmark
1 stars 0 forks source link

harness crashes intermittently #123

Closed van-smith closed 13 years ago

van-smith commented 13 years ago

While testing the fix for bug #74, numerous Java crash logs were generated for both SUTs, Llano and Brazos. Additionally, one of these crashes halted the batch file on the Llano SUT.

Here is the relevant information I recorded from bug #74:

I ran 100 iterations on both the Llano and Brazos SUTS where this failure occurs most frequently (although it is important to note that the failure has been seen on other SUTs).

There were four result XML files on the Llano SUT that recorded failures -- they occurred as two pairs of failures, with one failure causing the next. In both cases, the uninstall script failure triggered the install script failure, so it appears my fix worked for this run.

The eight Java crashes apparently only triggered one script failure, the one that occurred at 10:46.

For the Brazos SUT, there were three Java crashes and zero result files that indicated failure.

I will upload to Z the result XML files, the Java crash log files and the batch files within five minutes.

ghost commented 13 years ago

We need to have a phone meeting to discuss this.

van-smith commented 13 years ago

Please attempt to replicate the issue on your Brazos SUT.

ghost commented 13 years ago

Can we please discuss this on the phone? I do not understand what you mean by "three Java crashes". Was it the JVM that terminated? Completely? Was it an exception being reported which ceased that thread's execution?

If the JVM terminated, all OPBM processing ceases (obviously). There would be no programmatic indication of failure by OPBM in the Xml files (except manifest.xml which would report that the last worklet being run did not complete in the control section) because OPBM was never given information to log about anything failing, but would've simply been terminated itself. However, in such a case its parent (the JVM) would've logged its own failure, and at least noted that of the running process (OPBM).

van-smith commented 13 years ago

Rick, the GitHub issue tracker is not a chat forum. Send me an email if you want to chat. I'm calling you now.

van-smith commented 13 years ago

While attempting to duplicate the issue on the Core i5, the harness apparently froze while uninstalling Reader. However, when I repositioned the Reader window, the script continued.

During one of the subsequent installs, the installation window did not receive focus properly. Once I gave the window focus, the script continued.

The script later stalled again on the uninstall process and it was clearly a focus issue. I minimized Windows File Manager, from where I launched the tests, and the script continued until it got to the step to dismiss the completed uninstall message. Clicking on the window allowed the script to continue.

I'm running this test over RDP.

van-smith commented 13 years ago

Created bug #125 for the Adobe Reader install/uninstall focus issues.

van-smith commented 13 years ago

In the test above, I was attempting to install/uninstall Reader 100x. Since the script continued to stall due to focus issues, I terminated the batch file.

There were no Java crash logs, but the script was only able to complete 24 install/uninstall pairs.

I will now reboot the system and run the same test locally instead of over RDP.

van-smith commented 13 years ago

The Core i5 SUT completed 100 iterations of the install/uninstall process when the tests were run locally. The focus issue appears to be RDP related.

There were no Java crash logs generated.

The difference between this successful run and the unsuccessful runs on Llano and Brazos are:

I will now run the 100x test script on Llano after I install JDK 1.7.0 (64-bit).

van-smith commented 13 years ago

There are strange focus issues on the Llano SUT, but the scripts appear to be running properly so far.

van-smith commented 13 years ago

A Microsoft OneNote dialog appeared several times during testing. I'm not sure what triggered it.

However, there were no Java crash log files when running with HotSpot 1.7.0.

There also were no failure out of 100x runs. I am closing this bug with the resolution that HotSpot v1.7.0, but marking the bug as revisit because of the OneNote dialog and the lack of backwards compatibility.