search

phetsims / aqua

Automatic QUality Assurance
MIT License
2 stars 4 forks source link

Better error reporting for window.location probably changed (onbeforeunload handler) #173

Closed zepumph closed 1 year ago

zepumph commented 1 year ago

From https://github.com/phetsims/special-ops/issues/234, @jonathanolson and I want to investigate improving the experience for this.

A few ideas:

  1. ~Report the whole console logging with the error~
  2. ~See if console.error() occurs and treat those as Errors.~
  3. See if the puppeteer or parent frame (sounds hard) can give us more information.
  4. See if failIfNoLoad helps here.
  5. ~We think that much of this is from a puppeteer client closing after 15 minutes, what if instead it was every hour or 2?~
zepumph commented 1 year ago

@jonathanolson any worries about the following patch? 

Subject: [PATCH] I think we want this.
---
Index: js/server/Test.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/server/Test.js b/js/server/Test.js
--- a/js/server/Test.js (revision 9d01c74386ae4318b820325b3bd7db0ab294786d)
+++ b/js/server/Test.js (date 1682709370398)
@@ -202,7 +202,7 @@
     let url;

     if ( this.type === 'sim-test' ) {
-      url = `sim-test.html?url=${encodeURIComponent( `${baseURL}/${this.url}` )}`;
+      url = `sim-test.html?failIfNoLoad&url=${encodeURIComponent( `${baseURL}/${this.url}` )}`;

       if ( this.queryParameters ) {
         url += `&simQueryParameters=${encodeURIComponent( this.queryParameters )}`;
@@ -215,7 +215,7 @@
       url = `pageload-test.html?url=${encodeURIComponent( `${baseURL}/${this.url}` )}`;
     }
     else if ( this.type === 'wrapper-test' ) {
-      url = `wrapper-test.html?url=${encodeURIComponent( `${baseURL}/${this.url}` )}`;
+      url = `wrapper-test.html?failIfNoLoad&url=${encodeURIComponent( `${baseURL}/${this.url}` )}`;
     }
     if ( this.testQueryParameters ) {
       url = `${url}&${this.testQueryParameters}`;
zepumph commented 1 year ago

@jonathanolson and I talked about two more potential sources of this onbeforeunload error. Let's investigate both approaches.

zepumph commented 1 year ago

This test shows that we are in fact not getting the onbeforeunload from puppeteer whatsoever: puppeteerOnbeforeunloadTest.zip

It isn't until I add this patch that I begin receiving any onbeforeunload events both in the parent and in the iframe. So perhaps the puppeteer closing as the problem was a false assumption on my part. 

Subject: [PATCH] do that unload
---
Index: js/common/browserPageLoad.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/common/browserPageLoad.js b/js/common/browserPageLoad.js
--- a/js/common/browserPageLoad.js  (revision f07ceb27412350e2c82214dd329f63c260d0f16f)
+++ b/js/common/browserPageLoad.js  (date 1683048882674)
@@ -108,7 +108,7 @@
       timeout: options.gotoTimeout
     } );
     const result = await promise;
-    !page.isClosed() && await page.close();
+    !page.isClosed() && await page.close( { runBeforeUnload: true } );

     // If we created a temporary browser, close it
     ownsBrowser && await browser.close();
zepumph commented 1 year ago

@jonathanolson, after https://github.com/phetsims/aqua/issues/177 and a pull, you can use https://sparky.colorado.edu/continuous-testing/aqua/html/continuous-report.html?maxColumns=10&showBeforeUnloadErrors&filterString=window.location to see all the window.location errors. Yay!

zepumph commented 1 year ago

In https://github.com/phetsims/aqua/commit/1117c607b702e1535da21c094ef7d2434502a252 I changed it so that by default we don't see window.location errors. Feel free to change the default of that error, but in general that seems like an error for you and me to note, and not any others.

zepumph commented 1 year ago

We finally got our first logging from out new node client (https://github.com/phetsims/aqua/issues/178):

``` buoyancy : pan-and-zoom-fuzz : unbuilt http://127.0.0.1/continuous-testing/ct-snapshots/1683757090083/buoyancy/buoyancy_en.html?continuousTest=%7B%22test%22%3A%5B%22buoyancy%22%2C%22pan-and-zoom-fuzz%22%2C%22unbuilt%22%5D%2C%22snapshotName%22%3A%22snapshot-1683757090083%22%2C%22timestamp%22%3A1683757673160%7D&brand=phet&ea&fuzz&fuzzPointers=2&supportsPanAndZoom=true Query: brand=phet&ea&fuzz&fuzzPointers=2&supportsPanAndZoom=true Unloaded frame before complete, window.location probably changed [URL] http://127.0.0.1/continuous-testing/aqua/html/sim-test.html?url=..%2F..%2Fct-snapshots%2F1683757090083%2Fbuoyancy%2Fbuoyancy_en.html&simQueryParameters=brand%3Dphet%26ea%26fuzz%26fuzzPointers%3D2%26supportsPanAndZoom%3Dtrue&testInfo=%7B%22test%22%3A%5B%22buoyancy%22%2C%22pan-and-zoom-fuzz%22%2C%22unbuilt%22%5D%2C%22snapshotName%22%3A%22snapshot-1683757090083%22%2C%22timestamp%22%3A1683757673160%7D [NAVIGATED] http://127.0.0.1/continuous-testing/aqua/html/sim-test.html?url=..%2F..%2Fct-snapshots%2F1683757090083%2Fbuoyancy%2Fbuoyancy_en.html&simQueryParameters=brand%3Dphet%26ea%26fuzz%26fuzzPointers%3D2%26supportsPanAndZoom%3Dtrue&testInfo=%7B%22test%22%3A%5B%22buoyancy%22%2C%22pan-and-zoom-fuzz%22%2C%22unbuilt%22%5D%2C%22snapshotName%22%3A%22snapshot-1683757090083%22%2C%22timestamp%22%3A1683757673160%7D [NAVIGATED] about:blank [NAVIGATED] http://127.0.0.1/continuous-testing/ct-snapshots/1683757090083/buoyancy/buoyancy_en.html?continuousTest=%7B%22test%22%3A%5B%22buoyancy%22%2C%22pan-and-zoom-fuzz%22%2C%22unbuilt%22%5D%2C%22snapshotName%22%3A%22snapshot-1683757090083%22%2C%22timestamp%22%3A1683757673160%7D&brand=phet&ea&fuzz&fuzzPointers=2&supportsPanAndZoom=true [CONSOLE] enabling assert [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost [CONSOLE] WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost [CONSOLE] WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost [CONSOLE] WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost [CONSOLE] continuous-test-load [CONSOLE] THREE.WebGLRenderer: Context Lost. [CONSOLE] THREE.WebGLRenderer: Context Lost. [CONSOLE] THREE.WebGLRenderer: Context Lost. [CONSOLE] THREE.WebGLRenderer: Context Lost. [CONSOLE] continuous-test-unload id: "Sparky Node Puppeteer" density : fuzz : unbuilt : listenerOrderRandom http://127.0.0.1/continuous-testing/ct-snapshots/1683755010459/density/density_en.html?continuousTest=%7B%22test%22%3A%5B%22density%22%2C%22fuzz%22%2C%22unbuilt%22%2C%22listenerOrderRandom%22%5D%2C%22snapshotName%22%3A%22snapshot-1683755010459%22%2C%22timestamp%22%3A1683755709765%7D&brand=phet&ea&fuzz&listenerOrder=random Query: brand=phet&ea&fuzz&listenerOrder=random Unloaded frame before complete, window.location probably changed [URL] http://127.0.0.1/continuous-testing/aqua/html/sim-test.html?url=..%2F..%2Fct-snapshots%2F1683755010459%2Fdensity%2Fdensity_en.html&simQueryParameters=brand%3Dphet%26ea%26fuzz%26listenerOrder%3Drandom&testInfo=%7B%22test%22%3A%5B%22density%22%2C%22fuzz%22%2C%22unbuilt%22%2C%22listenerOrderRandom%22%5D%2C%22snapshotName%22%3A%22snapshot-1683755010459%22%2C%22timestamp%22%3A1683755709765%7D [NAVIGATED] http://127.0.0.1/continuous-testing/aqua/html/sim-test.html?url=..%2F..%2Fct-snapshots%2F1683755010459%2Fdensity%2Fdensity_en.html&simQueryParameters=brand%3Dphet%26ea%26fuzz%26listenerOrder%3Drandom&testInfo=%7B%22test%22%3A%5B%22density%22%2C%22fuzz%22%2C%22unbuilt%22%2C%22listenerOrderRandom%22%5D%2C%22snapshotName%22%3A%22snapshot-1683755010459%22%2C%22timestamp%22%3A1683755709765%7D [NAVIGATED] about:blank [NAVIGATED] http://127.0.0.1/continuous-testing/ct-snapshots/1683755010459/density/density_en.html?continuousTest=%7B%22test%22%3A%5B%22density%22%2C%22fuzz%22%2C%22unbuilt%22%2C%22listenerOrderRandom%22%5D%2C%22snapshotName%22%3A%22snapshot-1683755010459%22%2C%22timestamp%22%3A1683755709765%7D&brand=phet&ea&fuzz&listenerOrder=random [CONSOLE] enabling assert [CONSOLE] listenerOrder random seed: 382912 [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: INVALID_OPERATION: useProgram: program not valid [CONSOLE] WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use [CONSOLE] WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost [CONSOLE] WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost [CONSOLE] WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost [CONSOLE] continuous-test-load [CONSOLE] THREE.WebGLRenderer: Context Lost. [CONSOLE] THREE.WebGLRenderer: Context Lost. [CONSOLE] THREE.WebGLRenderer: Context Lost. [CONSOLE] continuous-test-unload id: "Sparky Node Puppeteer" Snapshot from 5/10/2023, 3:43:30 PM ```
zepumph commented 1 year ago

@jonathanolson, I don't see a lot from the above. Do you think it is related to the "Context Lost" logging? Would you like to put a couple more logs in here to try to understand the problem a bit more?

zepumph commented 1 year ago

https://github.com/phetsims/mobius/commit/d38c78dbc64297666d52e3d80932158d238fd342 will most likely fix the ThreeJS WebGL for density and buoyancy. Those are the only window.location errors that we are seeing on the new Node client, which feels like a good reason to go all in on it over in https://github.com/phetsims/aqua/issues/178.

zepumph commented 1 year ago
zepumph commented 1 year ago

There have been no window.location errors on the new node client since https://github.com/phetsims/mobius/commit/d38c78dbc64297666d52e3d80932158d238fd342.

zepumph commented 1 year ago

Alright! the gross hacky ?showBeforeUnloadErrors is gone now. Closing