search

DevExpress / testcafe-browser-provider-electron

This is the Electron browser provider plugin for TestCafe.
https://testcafe.io
MIT License
49 stars 30 forks source link

Running tests with executable (problem with setting mainWindowUrl) #87

Closed EmiM closed 5 months ago

EmiM commented 2 years ago

Hello,

I was looking through issues but I did not find anything helpful for my problem. We were using this testcafe plugin to test project (js app). Everything works fine (well, maybe except reopening app in another test case but that's a different issue). Now I want to test the built (executable) version of the app. I'm trying to first make it work with .AppImage. I can try it with .exe but I assume that the problem will be the same.

The app starts but I have a problem with mainWindowUrl. The README (especially code snippets) suggests that it would be the same setting as for the js app. However it doesn't work for me.

This is the current configuration:

module.exports = {
  mainWindowUrl: `./dist/main/index.html#/`,
  electronPath: `./dist/OurAppName.AppImage`
}

This is the error I get:

ERROR Unable to open the "electron:../frontend" browser due to the following error:
e2e-tests: Error: The main window page at file:///.../dev/monorepo/packages/frontend/dist/main/index.html#/ was not loaded.
e2e-tests: Use the mainWindowUrl option to specify one of the following pages as the main window page:
e2e-tests: file:///tmp/.mount_Quiet-yOG9W1/resources/app.asar/dist/main/index.html#/

It's great that the output suggests the page I should use but tmp/.mount_Quiet-yOG9W1/ is being created when the app starts and I don't have access to this path at the configuration level.

Is there something that I don't understand? I don't see any examples in the repo of testing executable code. I would really appreciate the help.

dositec commented 2 years ago

Hello,

The URL that Electron loads during startup should match the value of "mainWindowUrl" option. Based on your log:

  1. "mainWindowUrl" is file:///.../dev/monorepo/packages/frontend/dist/main/index.html#/
  2. However, your app tries to load file:///tmp/.mount_Quiet-yOG9W1/resources/app.asar/dist/main/index.html#/ If you dynamically create a page for the app, you can also dynamically specify the start page path in testcafe options.
EmiM commented 2 years ago

What do you mean by "dynamically"? Can I pass a function in testcafe config somehow?

I have an access to electron's APPDIR env variable that contains this needed path however testcafe config is loaded before APPDIR (or any other information about app) is known.

VasilyStrelyaev commented 2 years ago

Yes, you can pass a function to the TestCafe config, and you can try to use a getter for late binding (since APPDIR is not available when the config is read):

get-main-window-url.js

function getMainWindowUrl() {
    return `${process.env.APPDIR}/dist/main/index.html#/`;
}

export default getMainWindowUrl;

.testcafe-electron-rc.js

import getMainWindowUrl from './get-main-window-url';

module.exports = {
   get mainWindowUrl() {
       return getMainWindowUrl();
   },
   electronPath: `./dist/OurAppName.AppImage`
}

test.js

import getMainWindowUrl from './get-main-window-url';

const MAIN_PAGE = getMainWindowUrl();

fixture `Fixture 1`
    .page `${MAIN_PAGE}`;

//...

fixture `Fixture 2`
    .page `${MAIN_PAGE}/some-page`;

Please let me know your results.

EmiM commented 2 years ago

Thank you for the response!

Unfortunately it still doesn't work. It's called too early.

I had to add setter to the getter from your suggestion because otherwise I was getting TypeError: Cannot set property mainWindowUrl of #<Object> which has only a getter

I printed a traceback in the mainWindowUrl getter. You can see that's called right at the start in get-config and only before starting electron process.

e2e-tests: > cross-env E2E_TEST=true NODE_ENV=development DEBUG='waggle*,quiet*,testcafe:*electron*' testcafe --color --hostname localhost -S --screenshots-full-page -p '${DATE}_${TIME}_${TEST_ID}.png' "electron:../frontend" "./**/*.e2e.ts"
e2e-tests: Trace: Here mainWindowUrl getter from testcafe config file://undefined/dist/main/index.html#/
e2e-tests:     at Object.get mainWindowUrl [as mainWindowUrl] (/home/me/dev/monorepo/packages/frontend/.testcafe-electron-rc.js:17:16)
e2e-tests:     at exports.default (/home/me/dev/monorepo/packages/e2e-tests/node_modules/testcafe-browser-provider-electron/lib/utils/get-config.js:45:16)
e2e-tests:     at BrowserProviderPluginHost._callee3$ (/home/me/dev/monorepo/packages/e2e-tests/node_modules/testcafe-browser-provider-electron/lib/index.js:177:62)
e2e-tests:     at tryCatch (/home/me/dev/monorepo/packages/e2e-tests/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:62:40)
e2e-tests:     at Generator.invoke [as _invoke] (/home/me/dev/monorepo/packages/e2e-tests/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:296:22)
e2e-tests:     at Generator.prototype.<computed> [as next] (/home/me/dev/monorepo/packages/e2e-tests/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:114:21)
e2e-tests:     at step (/home/me/dev/monorepo/packages/e2e-tests/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
e2e-tests:     at /home/me/dev/monorepo/packages/e2e-tests/node_modules/babel-runtime/helpers/asyncToGenerator.js:35:14
e2e-tests:     at new Promise (<anonymous>)
e2e-tests:     at new F (/home/me/dev/monorepo/packages/e2e-tests/node_modules/core-js/library/modules/_export.js:36:28)
e2e-tests:     at BrowserProviderPluginHost.<anonymous> (/home/me/dev/monorepo/packages/e2e-tests/node_modules/babel-runtime/helpers/asyncToGenerator.js:14:12)
e2e-tests:     at BrowserProviderPluginHost.openBrowser (/home/me/dev/monorepo/packages/e2e-tests/node_modules/testcafe-browser-provider-electron/lib/index.js:243:26)
e2e-tests:     at BrowserProvider.openBrowser (/home/me/dev/monorepo/packages/e2e-tests/node_modules/testcafe/src/browser/provider/index.ts:317:27)
e2e-tests:     at BrowserConnection._runBrowser (/home/me/dev/monorepo/packages/e2e-tests/node_modules/testcafe/src/browser/connection/index.ts:203:33)
e2e-tests:     at /home/me/dev/monorepo/packages/e2e-tests/node_modules/testcafe/src/browser/connection/index.ts:173:37
e2e-tests:     at processTicksAndRejections (internal/process/task_queues.js:79:11)

In the meantime I noticed that I don't have access to APPDIR on the test level so unfortunately this wouldn't work anyway.

I can see that electron-mocks.js has access to the app.getAppPath. If we know that we are testing built version of electron app and app dir will be created dynamically then I'm wondering why mainWindowUrl can't be passed as a relative path (e.g /dist/main/index.html#/) and resolved to a full path later with getAppPath?

The fixture's page value produces similar but worse problem because there is no way to access app's directory (value of getAppPath) from the test file.

kowalski commented 2 years ago

I've also gave it at shot and I've sort was able to make progress, but I'm definitely not happy with direction it took.

I was able to get through initial verification making local change to following file: node_modules/testcafe-browser-provider-electron/templates/hook.js.mustache:

(function () {
   const originalConfig = {{{CONFIG}}};
   const config = {
     ...originalConfig,
     get mainWindowUrl() {
       const mainPage = `file://${process.env.APPDIR}/resources/app.asar/dist/main/index.html#/`;
       return mainPage;
     },
     set mainWindowUrl(value) {}
   }
   require('module')._load({{{INJECTABLE_PATH}}})(config, {{{TEST_PAGE_URL}}})
})();

The reason I need to move mainWindowUrl getter in here, is that the electron-side code is rendered like this:

export default function (config, testPageUrl) {
    return renderTemplate(HOOK_TEMPLATE, {
        INJECTABLE_PATH: INJECTABLE_PATH,
        CONFIG:          JSON.stringify(config),
        TEST_PAGE_URL:   JSON.stringify(testPageUrl)
    });

(testcafe-browser-provider-electron/src/hook.js)

therefore the getter/setter defined in .testcafe-electron-rc.js is not transfered over to the electron-side.

This change allowed my to pass through verification during launching of TestCafe.

Unfortunately I got stuck in different place. In my tests I have:

fixture`Electron test`.page(getMainWindowUrl())

test('User can create new community, register and send few messages to general channel', async t => {
  ....

It appears that the page can only be specified during load of the module and testcafe loads this module before spawning electron, therefore the APPDIR is not known yet.

What would allow me to have this working, would be if it were possible to specify the page dynamically on runtime from within the body of test. This is because test is executed after the electron has been started, therefore the path is already known. Do you know a way to specify page later than upon module load?

EmiM commented 2 years ago

I looked at the testcafe examples and I found out that there is a way to specify a 'page' from within the test body - await t.navigateTo(getMainWindowUrl())

EmiM commented 2 years ago

I managed to run tests but only with hacking testcafe electron plugin code.

It's obviously not the solution.

EmiM commented 2 years ago

@VasilyStrelyaev I ended up forking the plugin and adding few changes to be able to set mainWindowUrl properly. However I still have a problem with setting test case's page url.

1) TypeError: Cannot read property 'browserConnection' of undefined
e2e-tests:       Browser: Electron 12.1.0 / Linux 0.0
e2e-tests:          137 |    isMultiBrowser: true,
e2e-tests:          138 |    openedBrowsers: {},
e2e-tests:          139 |
e2e-tests:          140 |    _getBrowserHelpers: function _getBrowserHelpers() {
e2e-tests:          141 |        var testRun =
e2e-tests:       _testRunTracker2.default.resolveContextTestRun();
e2e-tests:        > 142 |        var id = testRun.browserConnection.id;
e2e-tests:          143 |
e2e-tests:          144 |        return
e2e-tests:       ElectronBrowserProvider.openedBrowsers[id].helpers;

I checked what is going on in resolveContextTestRun and this.activeTestRuns is an empty object that's why it doesn't work. I also can't see addActiveTestRun being called at any time.

I would want to be able to pass a relative (to electron.app.getAppPath) path in a test file or at least access the app.getAppPath from the test case. Is there a way to achieve this right now?

Do you use a different form of communication than github issues? Like Gitter? I think it would be much easier to talk this way.

alexfi1in commented 2 years ago

Hi @EmiM,

Please share an example that demonstrates how you start your electron app and TestCafe tests. We will try to find a suitable solution for you.

EmiM commented 2 years ago

Hi, sorry for the late response.

I created a simple example based on electron-react-boilerplate: https://github.com/EmiM/electron-react-boilerplate

Running the test:

npm i
npm run package
npm run e2e

You will see the same mainWindowUrl error that we encountered in our setup. It's possible that I set the wrong mainWindowUrl but then again - I'm wondering what should it be and README doesn't explain it very well in my opinion.

Aleksey28 commented 2 years ago

Hi @EmiM

I ran your example and got another error:

MicrosoftTeams-image (3)

Please check your example again step-by-step.

EmiM commented 2 years ago

Sorry, I didn't predict that it will be run on Windows as I've been using your plugin only on Linux (AppImage file).

I pushed a fix with handling app path for 3 different platforms but it still doesn't work on Windows as I would expect. Electron produces Setup exe file for Windows. Using this file directly gave me:

PS C:\dev\electron-react-boilerplate\release\build> npm run e2e
> electron-react-boilerplate@ e2e C:\dev\electron-react-boilerplate
> testcafe "electron:." "test.e2e.ts"
ERROR Unable to open the "electron:." browser due to the following error:
Error: Unable to connect
    at NodeInspect._callee2$ (C:\dev\electron-react-boilerplate\node_modules\testcafe-browser-provider-electron\lib\node-inspect.js:123:39)
    at tryCatch (C:\dev\electron-react-boilerplate\node_modules\babel-runtime\node_modules\regenerator-runtime\runtime.js:62:40)
    at Generator.invoke [as _invoke] (C:\dev\electron-react-boilerplate\node_modules\babel-runtime\node_modules\regenerator-runtime\runtime.js:296:22)
    at Generator.prototype.<computed> [as next] (C:\dev\electron-react-boilerplate\node_modules\babel-runtime\node_modules\regenerator-runtime\runtime.js:114:21)
    at step (C:\dev\electron-react-boilerplate\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
    at C:\dev\electron-react-boilerplate\node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13

So I manually run Setup exe file and then set the full path to the installed exe: (...)AppData\\Local\\Programs\\electron-react-boilerplate\\ElectronReact.exe' as electronPath with (...)/AppData/Local/Programs/electron-react-boilerplate/resources/app.asar/dist/renderer/index.html as mainWindowUrl. It worked, test passed. Great but it produces more problems:

Aleksey28 commented 2 years ago

Thank you for the example. I reproduced this behavior. We will research it and update this thread once we have any news.

For the team: remove the electronPath property from .testcafe-electron-rc.js

github-actions[bot] commented 6 months ago

This issue has been automatically marked as stale because it has not had any activity for a long period. It will be closed and archived if no further activity occurs. However, we may return to this issue in the future. If it still affects you or you have any additional information regarding it, please leave a comment and we will keep it open.

github-actions[bot] commented 5 months ago

We're closing this issue after a prolonged period of inactivity. If it still affects you, please add a comment to this issue with up-to-date information. Thank you.