redhat-developer / vscode-extension-tester

ExTester: Your Essential UI Testing Companion for Visual Studio Code Extensions! Seamlessly execute UI tests with Selenium WebDriver, ensuring robustness and reliability in your extension development journey. Simplify UI testing for your VS Code extensions and elevate the quality of your user interface effortlessly.
Apache License 2.0
251 stars 70 forks source link

[🚀 Request] Provide a method to check whether an extension is activated #854

Open lordrip opened 1 year ago

lordrip commented 1 year ago

Context Currently, it seems that there's no clear indicator from VSCode that a given extension is fully activated on the workbench.

What is the feature you are missing? vscode-extension-tester to provide a method that will be executed to determine whether the extension is activated. As an example, for vscode-kaoto we're using a timeout of 5 seconds as a workaround to ensure that the underlying backend is loaded and the UI had the chance to communicate with it (related issue).

Could you provide some example of usage? The idea would be to have a method like the existing waitForWorkbench that allows passing a custom function to test against the extension.

    /**
     * Waits until the activation function returns true to signal that the extension is ready
     */
    async waitForExtensionActivation(timeout = 30000, activationFn: () => Promise<void>): Promise<void>;

In addition to that, we could also provide some context to the activationFn, for instance, the underlying WebDriver or the stdout and stderr if available to have a better way to ensure that the extension is ready. Another possibility could be to give access to the Output pane of VSCode.

const activationFn = (context, stdout, sterr) => {
    //The context might be the WebDriver or another relevant object from VSCode

    // interact with the stdout if there's an underlying backend for the extension.
}
mlorinc commented 1 year ago

If I understand your test case, you wait for the Kaoto editor to show up. If it is the case, then I would check periodically for current CustomEditor title or some other feature which identifies your Kaoto WebView. Regarding the linked issue, I would recommend adding openResource call for each driver.wait attempt. After applying changes, it might look like the following code snippet, however I have not tested it:

export async function openAndSwitchToKaotoFrame(workspaceFolder: string, fileNameToOpen: string, driver: WebDriver, checkNotDirty: boolean) {
        // try to open editor, I believe wait interval should be set as well
        await await VSBrowser.instance.driver.wait(async () => {
        await VSBrowser.instance.openResources(path.join(workspaceFolder, fileNameToOpen));
        return await switchToKaotoFrame(driver, checkNotDirty);
       }, ...);
}

export async function switchToKaotoFrame(driver: WebDriver, checkNotDirty: boolean) {
    let kaotoEditor = new CustomEditor(); // handle error.NoSuchElementError just in case
    if (await kaotoEditor.getTitle() !== "your_title") { return undefined }; // or any other feature/flag identifier, depending on your application
    if (checkNotDirty) {
        assert.isFalse(await kaotoEditor.isDirty(), 'The Kaoto editor should not be dirty when opening it.');
    }
    let kaotoWebview :WebView = await kaotoEditor.getWebView();
    await driver.wait(async () => {
        try {
            kaotoEditor = new CustomEditor();
            kaotoWebview = await kaotoEditor.getWebView();
            await kaotoWebview.switchToFrame();
            return true;
        } catch (exception) {
            console.log('failed to switch to frame ' + exception);
            return false;
        }
    }, 20000, 'Failed to switch to frame', 1000);
    return { kaotoWebview, kaotoEditor };
}

I know it does not exactly answer your question, but hopefully it will help you to find workaround for now.

lordrip commented 1 year ago

Cool, thanks for sharing @mlorinc, I'll give it a try and see what happens.