microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
163.99k stars 29.19k forks source link

Flaky Terminal smoke tests #216564

Open rebornix opened 4 months ago

rebornix commented 4 months ago

https://dev.azure.com/monacotools/Monaco/_build/results?buildId=279265&view=results

1) VSCode Smoke Tests (Electron) Terminal Terminal Tabs should reset the tab name to the default value when no name is provided: Error: Timeout: get element '.single-terminal-tab' after 20 seconds. at Code.poll (D:\a_work\1\s\test\automation\out\code.js:204:23) at async Code.waitForElement (D:\a_work\1\s\test\automation\out\code.js:163:16) at async Terminal.assertTabExpected (D:\a_work\1\s\test\automation\out\terminal.js:220:17) at async Terminal.assertSingleTab (D:\a_work\1\s\test\automation\out\terminal.js:158:9) at async Context. (out\areas\terminal\terminal-tabs.test.js:42:13)

rebornix commented 4 months ago

https://dev.azure.com/monacotools/Monaco/_build/results?buildId=279278&view=results

1) VSCode Smoke Tests (Electron) Terminal Terminal splitCwd "after all" hook for "should inherit cwd when split and update the tab description - alt click": Error: Timeout: is active element ' .monaco-editor[data-uri$="settings.json"] textarea' after 20 seconds. at Code.poll (/mnt/vss/_work/1/s/test/automation/out/code.js:204:23) at async Code.waitForActiveElement (/mnt/vss/_work/1/s/test/automation/out/code.js:166:9) at async Editor.waitForEditorFocus (/mnt/vss/_work/1/s/test/automation/out/editor.js:68:9) at async SettingsEditor.openUserSettingsFile (/mnt/vss/_work/1/s/test/automation/out/settings.js:50:9) at async SettingsEditor.clearUserSettings (/mnt/vss/_work/1/s/test/automation/out/settings.js:41:9) at async Context. (out/areas/terminal/terminal-splitCwd.test.js:23:13)

aiday-mar commented 3 months ago

Seeing flaky terminal smoke tests https://dev.azure.com/monacotools/Monaco/_build/results?buildId=283595&view=logs&j=260a512b-e5ca-5cf1-3f8e-20c0fa41efc9&t=ecad4960-5379-5108-563e-8a65e2414e29 as above as well as

VSCode Smoke Tests (Web)
       Terminal
         Terminal stickyScroll
           "after all" hook for "should support multi-line prompt":
     Error: Timeout: click ' .monaco-editor[data-uri$="settings.json"] .view-lines > .view-line:nth-child(1)' after 20 seconds.
      at Code.poll (/Users/runner/work/1/s/test/automation/out/code.js:204:23)
      at async Code.waitAndClick (/Users/runner/work/1/s/test/automation/out/code.js:154:9)
      at async Editor.waitForEditorFocus (/Users/runner/work/1/s/test/automation/out/editor.js:67:9)
      at async SettingsEditor.openUserSettingsFile (/Users/runner/work/1/s/test/automation/out/settings.js:50:9)
      at async SettingsEditor.clearUserSettings (/Users/runner/work/1/s/test/automation/out/settings.js:41:9)
      at async Context.<anonymous> (out/areas/terminal/terminal-stickyScroll.test.js:25:13)

and

VSCode Smoke Tests (Web)
       Terminal
         Terminal Shell Integration
           Write data-based tests
             "after all" hook in "Write data-based tests":
     Error: Timeout: click ' .monaco-editor[data-uri$="settings.json"] .view-lines > .view-line:nth-child(1)' after 20 seconds.
      at Code.poll (/Users/runner/work/1/s/test/automation/out/code.js:204:23)
      at async Code.waitAndClick (/Users/runner/work/1/s/test/automation/out/code.js:154:9)
      at async Editor.waitForEditorFocus (/Users/runner/work/1/s/test/automation/out/editor.js:67:9)
      at async SettingsEditor.openUserSettingsFile (/Users/runner/work/1/s/test/automation/out/settings.js:50:9)
      at async SettingsEditor.clearUserSettings (/Users/runner/work/1/s/test/automation/out/settings.js:41:9)
      at async Context.<anonymous> (out/areas/terminal/terminal-shellIntegration.test.js:92:17)
aiday-mar commented 3 months ago

Seeing a flaky terminal smoke test in https://dev.azure.com/monacotools/Monaco/_build/results?buildId=284183&view=logs&j=260a512b-e5ca-5cf1-3f8e-20c0fa41efc9&t=82df682a-a732-590f-d041-2bbd067871c9

1) VSCode Smoke Tests (Electron)
       Terminal
         Terminal Persistence
           detach/attach
             should support basic reconnection:
     Error: Timeout: get element '.quick-input-widget' after 20 seconds.
      at Code.poll (/Users/runner/work/1/s/test/automation/out/code.js:204:23)
      at async Code.waitForElement (/Users/runner/work/1/s/test/automation/out/code.js:163:16)
      at async QuickInput.waitForQuickInputClosed (/Users/runner/work/1/s/test/automation/out/quickinput.js:32:9)
      at async Terminal.runCommandWithValue (/Users/runner/work/1/s/test/automation/out/terminal.js:110:9)
      at async Context.<anonymous> (out/areas/terminal/terminal-persistence.test.js:41:17)
aiday-mar commented 3 months ago

Also failing in the following build https://dev.azure.com/monacotools/Monaco/_build/results?buildId=284227&view=logs&j=9833cdaa-2e92-5f8d-56a8-239fd18fd628&t=91895ea7-021d-579e-1de7-d01a9dae3e5b

1) vscode API - terminal
       Terminal
         environmentVariableCollection
           should have collection variables apply to terminals immediately after setting:
     Error: Timeout: ~c2~c1 should be printed after 20 seconds.
Did not pass accept function
      at poll (/Users/runner/work/1/s/extensions/vscode-api-tests/src/utils.ts:176:10)
      at Context.<anonymous> (/Users/runner/work/1/s/extensions/vscode-api-tests/src/singlefolder-tests/terminal.test.ts:755:5)
meganrogge commented 3 months ago
  1) should create new terminals in the active editor group via command
        2) should create new terminals in the active editor group via plus button
        3) should create a terminal in the editor area by default
      Terminal Input
        Auto replies
          4) should automatically reply to a custom entry
      Terminal Persistence
        detach/attach
          5) should support basic reconnection
          - should persist buffer content
      Terminal Profiles
        6) should launch the default profile
        7) should set the default profile to a contributed one
        8) should use the default contributed profile on panel open and for splitting
        9) should set the default profile
        10) should use the default profile on panel open and for splitting
        11) createWithProfile command should create a terminal with a profile
        12) createWithProfile command should create a terminal with a contributed profile

https://dev.azure.com/monacotools/Monaco/_build/results?buildId=285406&view=results

meganrogge commented 3 months ago

Build: https://dev.azure.com/monacotools/a6d41577-0fa3-498e-af22-257312ff0545/_build/results?buildId=285577 Changes: https://github.com/Microsoft/vscode/compare/e3b1073...e3b1073

Tyriar commented 3 months ago

Observations:

Tyriar commented 3 months ago
2024-07-31 06:03:49.312 [info] lifecycle (main): phase changed (value: 2)
2024-07-31 06:03:49.404 [info] lifecycle (main): phase changed (value: 3)
2024-07-31 06:03:49.436 [info] update#setState disabled
2024-07-31 06:03:49.436 [info] update#ctor - updates are disabled by the environment
2024-07-31 06:03:51.910 [info] lifecycle (main): phase changed (value: 4)
2024-07-31 06:03:55.874 [error] ptyHost terminated unexpectedly with code 0
2024-07-31 06:03:57.288 [error] ptyHost terminated unexpectedly with code 0
2024-07-31 06:03:58.496 [error] ptyHost terminated unexpectedly with code 0
2024-07-31 06:03:59.034 [error] ptyHost terminated unexpectedly with code 0
2024-07-31 06:03:59.465 [error] ptyHost terminated unexpectedly with code 0
2024-07-31 06:04:00.644 [error] ptyHost terminated unexpectedly with code 0
2024-07-31 06:04:01.122 [error] ptyHost terminated unexpectedly with code 0, giving up
2024-07-31 06:04:01.123 [error] [UtilityProcess type: ptyHost, pid: 199359]: crashed with code 134 and reason 'crashed'
2024-07-31 06:04:20.653 [warning] No ptyHost heartbeat after 6 seconds
2024-07-31 06:04:25.657 [error] No ptyHost heartbeat after 12 seconds