New Opera update dialog causes OPBM to fail

[jimmacd]

This is going to a nuance we will have to deal with. Given the latest mode of development with greatly accelerated release cadences (firefox for example), users are likely to encounter update dialogs far more often than in the past. This is going to cause the benchmark to fail until the update is dealt with. I don't know if this is simply a warning dialog or something more elegant but we need to deal with this in some way.

[van-smith]

Yes, this is an expected concern. At this point, since Opera needs to be preinstalled, the workaround is to accept the update and rerun OPBM.

However, automatic updates can be disabled within Opera: Settings | Preferences | Advanced | Security | Auto-update and select "Do not check for updates".

[jimmacd]

I just brought it up as something to keep in mind. I was on a continual set of runs checking on reliability (flawless by the way) and was surprised to look up and find it stopped. I figured it was worth mentioning.

[van-smith]

Thanks, Jim, your feedback truly is invaluable to us.

[van-smith]

BTW, I am going to reopen this issue until we have a fix implemented,

[ghost]

Partial fix applied, which may be a full fix if we begin installing Opera from the harness again: turns off auto-updates through operaprefs.ini.

[van-smith]

OPBM crashed while attempting to run the SunSpider test. The harness soft-hung at the failure. The last few lines in the CLI were:

Executing: IE Google V8 Executing: Opera SunSpider Exception in thread "OPBMStartupThread" java.lang.NullPointerException at opbm.benchmarks.BenchmarkManifestResults.computeResultsViewerTotalsAn dGenerateCSVFile(BenchmarkManifestResults.java:1470) at opbm.benchmarks.BenchmarkManifest.computeForResultsXml(BenchmarkManif est.java:2576) at opbm.benchmarks.BenchmarkManifest.run(BenchmarkManifest.java:1623) at opbm.Opbm$1$1.run(Opbm.java:632)

OPBM had been launched using the following command:

java -jar opbm.jar -scenario(9):opbm

The failure occurred on the first iteration.

The HUD read "Error was handled:" but nothing followed. CPU utilization was at 0% on the Llano SUT.

Clicking the Stop button resulted in many "Attempting to stop, please wait..." messages. I waited five minute before manually shutting down OPBM. Again, CPU utilization was at 0% on the Llano SUT.

After killing OPBM, I tried to launch Opera and encountered the "Update Opera" notification, so presumably this failure is due to bug #92.

This problem occurred with code pulled from the dev branch yesterday evening.

[van-smith]

After rebooting, I relaunched OPBM as described above. A similar exception occurred almost immediately:

atomCheckConflicts: IE9 Conflict Check atomCheckConflicts: Opera Conflict Check atomCheckConflicts: Word Conflict Check Executing: Install 7zip Exception in thread "OPBMStartupThread" java.lang.NullPointerException at opbm.benchmarks.BenchmarkManifestResults.computeResultsViewerTotalsAn dGenerateCSVFile(BenchmarkManifestResults.java:1470) at opbm.benchmarks.BenchmarkManifest.computeForResultsXml(BenchmarkManif est.java:2576) at opbm.benchmarks.BenchmarkManifest.run(BenchmarkManifest.java:1623) at opbm.Opbm$1$1.run(Opbm.java:632)

Consequently, this appears to go beyond bug #92.

[van-smith]

7zip failure reproduced with the latest pull.

[van-smith]

I deleted the user opbm directory and the 7zip failure continued to reproduce.

[ghost]

"The HUD read "Error was handled:" but nothing followed."

The HUD displays messages from AutoIt scripts, so the error was handled in the script. There are a few messages OPBM displays in the HUD itself, the pause between scripts where it's waiting for idle, the reboot messages, and the attempting to stop messages.

Once the null pointer error was encountered, there was nothing processing the benchmark thread any longer. It would never recover, which is why OPBM hung. If you are running using java.exe and seeing a command window, closing the command window will close OPBM and allow it to be restarted.

[ghost]

Which 7zip failure? A 7zip failure causing the opera update error?

[ghost]

Please post the "Gather Debug Info" file to the Z: drive on all bugs if possible, but definitely on those resulting from uncaught exceptions thrown, as I will be able to see and reproduce the exact conditions within the app which caused the failure, stepping through the code line-by-line.

[van-smith]

I reported the 7zip failure earlier in this bug incident.

There is no debug info to gather because OPBM is in an unrecoverable state and must be terminated via Task Manager.

[ghost]

One OPBM restarts, please gather debug info then. The files it gathers are persistent.

[van-smith]

The debug is on Z.

[van-smith]

I reran the benchmark on the same SUT, but used the GUI to launch the test. The failure did not manifest.

However, I encountered what appears to be the Java deadlock condition.

[ghost]

This issue should be corrected after the 8b8bd36 push.

[van-smith]

The current code appears to time out and terminate Opera if Opera does not launch. It appears to then relaunch Opera until the upgrade dialog does not appear.

This only appears to work intermittently. We have seen successes on the Core i5 SUT, but we hit this failure on the Brazos SUT using the code pulled down late on 10.30.

The debug info for the Brazos failure is on Z.

[van-smith]

The failure has occurred again on the Brazos SUT with the latest code pull.

[rick-pritchett]

Created code in the launchOpera function (operaCommon) to check for and close known extraneous windows (Update and Welcome). This check is implemented before the "timed" launch. Initial tests work as desired. Continuing to test.

[rick-pritchett]

This should also take care of the first window mentioned in bug #119 ("... Opera was closed improperly and wanted to know whether if should continue..."). Thanks, Van, for showing me how to make that window appear on demand.

[rick-pritchett]

Tests on dev laptop successful. Currently testing on rick-i5 and Atom SUTs.

[rick-pritchett]

Tests on rick-i5 and Atom successful. Pushed new operaCommon, operaInstall, operaSunSpider, operaKraken, and operaGoogleV8.

[van-smith]

Closing per Rick's direction.