DevExpress / testcafe

A Node.js tool to automate end-to-end web testing.
https://testcafe.io
MIT License
9.82k stars 671 forks source link

Edge occassionally shows a "set up/signin" modal after startup, causing testcafe to fail #8149

Closed christian-fischer closed 7 months ago

christian-fischer commented 7 months ago

What is your Scenario?

Using Edge with native automation enabled.

What is the Current behavior?

Occassionally [1] immediately after testcafe starts Edge, a modal appears regarding some profile setup/signin (see screenshot). This subsequently fails the testrun, as not even the http://localhost:4300/browser/connect/* url is loaded, due to the modal blocking the browser.

[1] sometimes as frequent as every second run, other times as rarely as every ~20th run

What is the Expected behavior?

I expect no modal to appear and that Edge loads the page I want to test.

What is the public URL of the test page? (attach your complete example)

Issue happens before the test page is even loaded. So for testing purposes using any page is fine, i.e. https://testcafe.io

What is your TestCafe test code?

fixture('test');
test('edge', async (tc) => {
  await tc.navigateTo('https://testcafe.io');
  await tc.expect(true).ok();
});

Your complete configuration file

// no specific configuration needed, fails with defaults

Your complete test report

Live mode is enabled.
TestCafe now watches source files and reruns
the tests once the changes are saved.

You can use the following keys in the terminal:
'Ctrl+S' - stops the test run;
'Ctrl+R' - restarts the test run;
'Ctrl+W' - enables/disables watching files;
'Ctrl+C' - quits live mode and closes the browsers.

Watching the following files:
  .\edge.test.js

Stopping TestCafe live mode...

Make changes to the source files or press Ctrl+R to restart the test run.
ERROR Unable to open the "edge:" browser due to the following error:

TypeError: Cannot read properties of undefined (reading 'Fetch')
    at NativeAutomationRequestPipeline.start (.\node_modules\testcafe\src\native-automation\request-pipeline\index.ts:387:28)
    at NativeAutomation.start (.\node_modules\testcafe\src\native-automation\index.ts:46:29)
    at BrowserProviderPluginHost._setupNativeAutomation (.\node_modules\testcafe\src\browser\provider\built-in\dedicated\chrome\index.js:63:32)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at BrowserProviderPluginHost.openBrowser (.\node_modules\testcafe\src\browser\provider\built-in\dedicated\chrome\index.js:109:13)
    at BrowserProvider.openBrowser (.\node_modules\testcafe\src\browser\provider\index.ts:326:9)
    at BrowserConnection._runBrowser (.\node_modules\testcafe\src\browser\connection\index.ts:279:13)

Type "testcafe -h" for help.

Screenshots

testcafe-edge-issue

Steps to Reproduce

  1. run testcafe edge edge.test.js -L
  2. observe the modal interfering with test execution

TestCafe version

3.2.0

Node.js version

18.12.0

Command-line arguments

testcafe edge edge.test.js

Browser name(s) and version(s)

Edge 122.0.2365.52

Platform(s) and version(s)

Windows 11 23H2

Other

No response

PavelMor25 commented 7 months ago

Hello @christian-fischer,

We can't reproduce the issue on our machines ( windows 10-11 / nodeJS 18-20 / Edge 122.0.2365.52-59 / testcafe 3.2 - 3.5).

Could you please try to reproduce the issue on other machines?

Perhaps, you can specify details that can help us reproduce the problematic behavior.

christian-fischer commented 7 months ago

Hi @PavelMor25 when I tried reproducing this, it only happened occasionally - so I had to execute the command several times to get one failing run due to the described issue. I'm not sure how edge determines when to show this modal dialog or not - maybe there are pre-requisites regarding the windows-account?

As testcafe controls the profile in native automation, maybe it's possible to set some edge policy? The HideFirstRunExperience policy sounds promising in this regard to suppress this modal dialog.

PavelMor25 commented 7 months ago

Hello @christian-fischer,

TestCafe runs Edge with a custom user profile to which we add some properties:

fre: This property stores information about whether the user has seen the offer to use Microsoft Edge features and capabilities. The value of has_user_seen_fre indicates whether the user has seen this information.

profiles: This property holds information about user profiles. In this case, the edge_implicitly_signed_in property indicates automatic sign-in to the Microsoft Edge account for the specified account type.

You can also try to view the profile file created by TestCafe and check if these properties are indeed present there.

Therefore, on launching Edge, this window should not be displayed. Unfortunately, since we cannot reproduce the issue at the moment, we are unable to assist you further.

github-actions[bot] commented 7 months ago

This issue was automatically closed because there was no response to our request for more information from the original author. Currently, we don't have enough information to take action. Please reach out to us if you find the necessary information and are able to share it. We are also eager to know if you resolved the issue on your own and can share your findings with everyone.