redhat-developer / vscode-extension-tester

ExTester: Your Essential UI Testing Companion for Visual Studio Code Extensions! Seamlessly execute UI tests with Selenium WebDriver, ensuring robustness and reliability in your extension development journey. Simplify UI testing for your VS Code extensions and elevate the quality of your user interface effortlessly.
Apache License 2.0
255 stars 70 forks source link

NoSuchSessionError #310

Closed AllanOricil closed 1 year ago

AllanOricil commented 3 years ago

Im trying to execute the following test and I can't get a TerminalView to work. The terminal opens, but for some reason the Promise never resolves to a terminalView. It is always going into the catch.

// import the webdriver and the high level browser wrapper
import { VSBrowser, WebDriver, Workbench, BottomBarPanel } from 'vscode-extension-tester';

// Create a Mocha suite
describe('My Test Suite', () => {
    let driver: WebDriver

    // initialize the browser and webdriver
    before(async () => {
        driver = VSBrowser.instance.driver;
    });

    // test whatever we want using webdriver, here we are just checking the page title
    it('My Test Case', async () => {
        new BottomBarPanel().openTerminalView()
        .then((terminalView) => {
            terminalView.getChannelNames().then((names) => {
                terminalView.executeCommand("ls")
                .then(() => {
                    new Workbench().executeCommand('SFDX: Open Salesforce Query Editor')
                    .then(() => {
                        console.log('hi');
                    })
                })
                .catch((e) => {
                    console.log(e);
                })
            })

        })
        .catch((e) => {
            console.error(e);
        })

    });
});

this is the output I receive, when I launch the test with a debugger attached

C:\Users\allan_000\workspace\soe2 [setup-tests ↑2 +0 ~6 -1 !]>  c:; cd 'c:\Users\allan_000\workspace\soe2'; ${env:NODE_OPTIONS}='--require "c:/Users/allan_000/AppData/Local/Programs/Microsoft VS Code/resources/app/extensions/ms-vscode.js-debug/src/bootloader.bundle.js" --inspect-publish-uid=http'; ${env:VSCODE_INSPECTOR_OPTIONS}='{"inspectorIpc":"\\\\.\\pipe\\node-cdp.19936-12.sock","deferredMode":false,"waitForDebugger":"","execPath":"C:\\Program Files\\nodejs\\node.exe","onlyEntrypoint":false,"autoAttachMode":"always","fileCallback":"C:\\Users\\ALLAN_~1\\AppData\\Local\\Temp\\node-debug-callback-1a1286b69804aaf3"}'; & 'C:\Program Files\nodejs\node.exe' '.\node_modules\vscode-extension-tester\out\cli.js' 'setup-and-run' 'C:\Users\allan_000\workspace\soe2/out/test/*.js' '--mocha_config' 'C:\Users\allan_000\workspace\soe2/src/test/.mocharc-debug.js' '-y' 
Debugger attached.
VS Code exists in local cache, skipping download
ChromeDriver 89.0.4389.23 exists in local cache, skipping download
Debugger attached.
This extension consists of 1978 files, out of which 278 are JavaScript files. For performance reasons, you should bundle your extension: https://aka.ms/vscode-bundle-extension . You should also exclude unnecessary files by adding them to your .vscodeignore: https://aka.ms/vscode-vscodeignore
 DONE  Packaged: C:\Users\allan_000\workspace\soe2\salesforce-soql-editor-1.8.0.vsix (1978 files, 238.2MB)
Waiting for the debugger to disconnect...
Installing extensions...
(node:15700) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `Code --trace-deprecation ...` to show where the warning was created)
Extension 'salesforce-soql-editor-1.8.0.vsix' was successfully installed.
Debugger attached.
Waiting for the debugger to disconnect...
Installing extensions...
(node:24020) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `Code --trace-deprecation ...` to show where the warning was created)
Extension 'api-handler.vsix' was successfully installed.
Debugger attached.
Waiting for the debugger to disconnect...

Writing code settings to C:\Users\allan_000\workspace\soe2\test-resources\settings\User\settings.json
Launching browser...

[4604:0627/190812.626:ERROR:node_bindings.cc(273)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[4604:0627/190812.627:ERROR:node_bindings.cc(273)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[4604:0627/190812.627:ERROR:node_bindings.cc(273)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[4604:0627/190812.627:ERROR:node_bindings.cc(273)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.
[4604:0627/190812.627:ERROR:node_bindings.cc(273)] Most NODE_OPTIONs are not supported in packaged apps. See documentation for more details.

DevTools listening on ws://127.0.0.1:64079/devtools/browser/4404f1b7-ba98-46a5-b01e-acdc51b00d63
Warning: 'disable-background-networking' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'disable-backgrounding-occluded-windows' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'disable-client-side-phishing-detection' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'disable-default-apps' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'disable-dev-shm-usage' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'disable-hang-monitor' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'disable-popup-blocking' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'disable-prompt-on-repost' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'disable-sync' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'enable-automation' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'enable-blink-features' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'enable-logging' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'log-level' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'first-run' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'sandbox' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'service-autorun' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'password-store' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'remote-debugging-port' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'test-type' is not in the list of known options, but still passed to Electron/Chromium.
Warning: 'use-mock-keychain' is not in the list of known options, but still passed to Electron/Chromium.
[main 2021-06-27T22:08:13.159Z] update#setState idle
[4604:0627/190813.424:INFO:CONSOLE(61)] "The vm module of Node.js is deprecated in the renderer process and will be removed.", source: electron/js2c/renderer_init.js (61)
[4604:0627/190814.685:INFO:CONSOLE(61)] "The vm module of Node.js is deprecated in the renderer process and will be removed.", source: electron/js2c/renderer_init.js (61)
[4604:0627/190815.422:INFO:CONSOLE(62)] "%c[Extension Host] %c(node:21736) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `Code --trace-deprecation ...` to show where the warning was created) color: blue color: ", source: file:///C:/Users/allan_000/workspace/soe2/test-resources/VSCode-win32-x64-archive/resources/app/out/vs/workbench/workbench.desktop.main.js (62)
Browser ready in 5698 ms
Launching tests...
  My Test Suite
[4604:0627/190820.351:INFO:CONSOLE(62)] "%c[Extension Host] %cMessage View AWS Toolkit Logs didn't get externalized correctly. color: blue color: ", source: file:///C:/Users/allan_000/workspace/soe2/test-resources/VSCode-win32-x64-archive/resources/app/out/vs/workbench/workbench.desktop.main.js (62)
[4604:0627/190820.351:INFO:CONSOLE(62)] "%c[Extension Host] %cMessage Connecting... didn't get externalized correctly. color: blue color: ", source: file:///C:/Users/allan_000/workspace/soe2/test-resources/VSCode-win32-x64-archive/resources/app/out/vs/workbench/workbench.desktop.main.js (62)
[4604:0627/190820.351:INFO:CONSOLE(62)] "%c[Extension Host] %cMessage Connecting... didn't get externalized correctly. color: blue color: ", source: file:///C:/Users/allan_000/workspace/soe2/test-resources/VSCode-win32-x64-archive/resources/app/out/vs/workbench/workbench.desktop.main.js (62)
[4604:0627/190820.352:INFO:CONSOLE(62)] "%c[Extension Host] %cMessage Add a region to AWS Explorer... didn't get externalized correctly. color: blue color: ", source: file:///C:/Users/allan_000/workspace/soe2/test-resources/VSCode-win32-x64-archive/resources/app/out/vs/workbench/workbench.desktop.main.js (62)
[4604:0627/190820.352:INFO:CONSOLE(62)] "%c[Extension Host] %cMessage Click here to add a region to AWS Explorer. didn't get externalized correctly. color: blue color: ", source: file:///C:/Users/allan_000/workspace/soe2/test-resources/VSCode-win32-x64-archive/resources/app/out/vs/workbench/workbench.desktop.main.js (62)
[4604:0627/190820.529:INFO:CONSOLE(62)] "%c[Extension Host] %cMessage AWS Toolkit didn't get externalized correctly. color: blue color: ", source: file:///C:/Users/allan_000/workspace/soe2/test-resources/VSCode-win32-x64-archive/resources/app/out/vs/workbench/workbench.desktop.main.js (62)
[4604:0627/190820.604:INFO:CONSOLE(62)] "%c[Extension Host] %cMessage AWS Remote Invocations didn't get externalized correctly. color: blue color: ", source: file:///C:/Users/allan_000/workspace/soe2/test-resources/VSCode-win32-x64-archive/resources/app/out/vs/workbench/workbench.desktop.main.js (62)
    1) My Test Case

Shutting down the browser
Uninstalling vscode-extension-tester.api-handler...
Extension 'vscode-extension-tester.api-handler' was successfully uninstalled!
Debugger attached.
Waiting for the debugger to disconnect...

  0 passing (23s)
  1 failing

  1) My Test Suite
       My Test Case:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (C:\Users\allan_000\workspace\soe2\out\test\query-editor-spec.js)      at listOnTimeout (internal/timers.js:531:17)
      at processTimers (internal/timers.js:475:7)

Waiting for the debugger to disconnect...
C:\Users\allan_000\workspace\soe2\node_modules\selenium-webdriver\lib\promise.js:2626
        throw error;
        ^

NoSuchSessionError: invalid session id
    at Object.throwDecodedError (C:\Users\allan_000\workspace\soe2\node_modules\selenium-webdriver\lib\error.js:514:15)
    at parseHttpResponse (C:\Users\allan_000\workspace\soe2\node_modules\selenium-webdriver\lib\http.js:519:13)
    at C:\Users\allan_000\workspace\soe2\node_modules\selenium-webdriver\lib\http.js:441:30
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
From: Task: WebElement.findElement(By(xpath, .//li[a/span/@aria-label="Editor Layout"]))
    at Driver.schedule (C:\Users\allan_000\workspace\soe2\node_modules\selenium-webdriver\lib\webdriver.js:807:17)
    at ContextMenu.schedule_ (C:\Users\allan_000\workspace\soe2\node_modules\selenium-webdriver\lib\webdriver.js:2010:25)
    at ContextMenu.findElement (C:\Users\allan_000\workspace\soe2\node_modules\selenium-webdriver\lib\webdriver.js:2057:17)
    at new AbstractElement (C:\Users\allan_000\workspace\soe2\node_modules\monaco-page-objects\out\components\AbstractElement.js:39:48)
    at new MenuItem (C:\Users\allan_000\workspace\soe2\node_modules\monaco-page-objects\out\components\menu\MenuItem.js:17:1)
    at new ContextMenuItem (C:\Users\allan_000\workspace\soe2\node_modules\monaco-page-objects\out\components\menu\ContextMenu.js:110:9)
    at ContextMenu.<anonymous> (C:\Users\allan_000\workspace\soe2\node_modules\monaco-page-objects\out\components\menu\ContextMenu.js:50:38)
    at Generator.next (<anonymous>)
    at fulfilled (C:\Users\allan_000\workspace\soe2\node_modules\monaco-page-objects\out\components\menu\ContextMenu.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  name: 'NoSuchSessionError',
  remoteStacktrace: 'Backtrace:\n' +
    '\tOrdinal0 [0x00C0E7D3+124883]\n' +
    '\tOrdinal0 [0x00C0E7B1+124849]\n' +
    '\tGetHandleVerifier [0x00E58530+193488]\n' +
    '\tGetHandleVerifier [0x00E74D23+310211]\n' +
    '\tGetHandleVerifier [0x00E75D8E+314414]\n' +
    '\tGetHandleVerifier [0x00E75D19+314297]\n' +
    '\tGetHandleVerifier [0x00F331EC+1089676]\n' +
    '\tGetHandleVerifier [0x00F317C9+1082985]\n' +
    '\tGetHandleVerifier [0x00F313A3+1081923]\n' +
    '\tGetHandleVerifier [0x0102D9FD+2115741]\n' +
    '\tOrdinal0 [0x00D5B82E+1488942]\n' +
    '\tOrdinal0 [0x00CF5A7D+1071741]\n' +
    '\tOrdinal0 [0x00CF559B+1070491]\n' +
    '\tOrdinal0 [0x00CF54B1+1070257]\n' +
    '\tOrdinal0 [0x00D2FF53+1310547]\n' +
    '\tBaseThreadInitThunk [0x76C1FA29+25]\n' +
    '\tRtlGetAppContainerNamedObjectPath [0x76FE7A9E+286]\n' +
    '\tRtlGetAppContainerNamedObjectPath [0x76FE7A6E+238]\n'
}
C:\Users\allan_000\workspace\soe2 [setup-tests ↑2 +0 ~6 -1 !]> 
AllanOricil commented 3 years ago

This also does not work

// import the webdriver and the high level browser wrapper
import { VSBrowser, WebDriver, Workbench, BottomBarPanel, TerminalView } from 'vscode-extension-tester';

// Create a Mocha suite
describe('My Test Suite', () => {
    let driver: WebDriver

    // initialize the browser and webdriver
    before(async () => {
        driver = VSBrowser.instance.driver;
    });

    // test whatever we want using webdriver, here we are just checking the page title
    it('My Test Case', async () => {
        try{
            const bottomBarPanel : BottomBarPanel = new BottomBarPanel();
            const terminalView : TerminalView  = await bottomBarPanel.openTerminalView()
        }catch(e){
            console.log(e);
        }
    });
});
jrichter1 commented 3 years ago

From the logs, I can see the test actually failed with a timeout

1) My Test Suite
       My Test Case:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (C:\Users\allan_000\workspace\soe2\out\test\query-editor-spec.js)      at listOnTimeout (internal/timers.js:531:17)
      at processTimers (internal/timers.js:475:7)

Mocha test cases by default have a 2 second hard timeout. That's hardly enough for most UI tests, so you can set it to whatever you like:

it('test case', () => {
  // test block
}).timeout(timeoutInMilliseconds)

As for the NoSuchSessionError, there is a forced shutdown after all tests are complete. The error happens if there still is an unresolved promise in the webdriver queue at that time. Usually it means some asynchronous task leaked from a test case unawaited. There might be a way to purge the queue before shutdown, which would keep the error from appearing again. Some might say the error has some debug value, though.

jrichter1 commented 3 years ago

BTW the first test case doesn't actually do much, because you need to return promises in order to chain them. As for the second attempt, I have no idea what could have gone wrong.

AllanOricil commented 3 years ago

@jrichter1 When I used the async/await approach it threw the same error. When the test starts I can see the terminal is opened, but when the test reaches this line terminalView.executeCommand("ls") , terminalView is undefined somehow and I also get that exception. I can't understand if Im doing something wrong or it is a bug because the doc does not have a proper example :/ If you or someone else could help me I would appreciate :)

jrichter1 commented 3 years ago

The async/await code looks like it shouldn't run into any problems, it runs completely fine when I'm trying it.

Given you are experiencing the same error as in the previous attempt, make sure you re-compile the .ts test files into .js every time you change and run them.

As far as the then chain is concerned, you can only chain promises. And if you don't return anything from the then callbacks, it can't chain on void type. So your code basically calls the first promise, and then doesn't wait for the callbacks to chain. Mocha also implicitly only waits for promises that are either awaited, or returned. In the end, a couple of return statements should make it work.

it('My Test Case', async () => {
return new BottomBarPanel().openTerminalView()
        .then((terminalView) => {
            return terminalView.getChannelNames().then((names) => {
                return terminalView.executeCommand("ls")
                .then(() => {
                    return new Workbench().executeCommand('SFDX: Open Salesforce Query Editor')
                    .then(() => {
                        console.log('hi');
                    })
                })
                .catch((e) => {
                    console.log(e);
                })
            })

        })
        .catch((e) => {
            console.error(e);
        })
    });
AllanOricil commented 3 years ago

Oh I see. I have not tried to add a return there. Thanks @jrichter1. I will try and later come back with the results