eclipse-theia / theia

Eclipse Theia is a cloud & desktop IDE framework implemented in TypeScript.
http://theia-ide.org
Eclipse Public License 2.0
20.08k stars 2.5k forks source link

Intermittent Playwright test failures in `theia-notebook-editor.test` #14450

Open planger opened 1 week ago

planger commented 1 week ago

I'm monitoring the stability of the Playwright tests and observed that there are a number of intermittent, recurring test failures at the theia-notebook-editor.test.ts.

Error:   1) ../../src/tests/theia-notebook-editor.test.ts:98:9 › Theia Notebook Editor interaction › should split cell 
    Error: elementHandle.waitForElementState: Element is not attached to the DOM
    Call log:
      -   waiting for element to be visible

      111 |         */
      112 |         const line = await cell.editor.lineByLineNumber(1);
    > 113 |         await line?.waitForElementState('visible');
          |                     ^
      114 |         await line?.click();
      115 |         await line?.press('ArrowRight');
      116 |

        at /home/runner/work/theia/theia/examples/playwright/src/tests/theia-notebook-editor.test.ts:113:21

It is not alarmingly frequent, but it might be good to look into it anyway to avoid false-positives.

planger commented 1 week ago

@dhuebner Would you be able to see if we can avoid these failures? Thanks a lot in advance!

dhuebner commented 1 week ago

@planger Sure, I will take a look. Thank you for reporting!

dhuebner commented 1 week ago

@planger I was not able to reproduce it locally :( I've prepared a PR #14457 that removes the visible check and adds some logging to see the line Element state. If the test fails again, I hope I can narrow down the cause. In the test reports it is obvious that all cell-editor lines are visible when the test was failing. See attached report.

playwright-test-results-3.zip