DevExpress / testcafe

A Node.js tool to automate end-to-end web testing.
https://testcafe.io
MIT License
9.78k stars 661 forks source link

Cannot run testcafe tests against latest portable edge in headless mode #8200

Closed Makavelic closed 1 day ago

Makavelic commented 1 month ago

What is your Scenario?

It is not possible to run testcafe tests against a local version of edge.

What is the Current behavior?

Edge cannot run

What is the Expected behavior?

Edge should run without errors

What is the public URL of the test page? (attach your complete example)

Any project will do, you can reproduce by pointing to a local version of edge:

testcafe "path:d:\edgeportable\msedge.exe --headless" test.js

Note: Even if you try running headless mode locally without testcafe you get an error.

If you try and force the new headless=new feature it does work manually but not in testcafe.

Ex: testcafe "path:d:\edgeportable\msedge.exe --headless=new" test.js

What is your TestCafe test code?

None needed

Your complete configuration file

No response

Your complete test report

No response

Screenshots

No response

Steps to Reproduce

  1. Run cli testcafe "path:d:\edgeportable\msedge.exe --headless" test.js
  2. Notice Error
  3. Run cli testcafe "path:d:\edgeportable\msedge.exe --headless=new" test.js
  4. Notice how the tests stalls indefinitely (browser init never finishes)

Note: The following paths don't work either:

edge:d:\edgeportable\msedge.exe:headless (error) edge:d:\edgeportable\msedge.exe:headless=new (runs without headless but doesn't work)

Note: this works on older versions of edge like 118.

TestCafe version

3.6.0

Node.js version

18.18.2

Command-line arguments

testcafe "path:d:\edgeportable\msedge.exe --headless" test.js

Browser name(s) and version(s)

Edge 125

Platform(s) and version(s)

Windows 11

Other

No response

PavelMor25 commented 1 month ago

Hello @Makavelic,

Please clarify whether you used the following command to run tests:

testcafe "path:d:\edgeportable\msedge.exe --headless"  test.js

In your examples, the --headless and --headless=new options are outside the quotation marks. Therefore, they are parsed as TestCafe options rather than browser options.

Makavelic commented 1 month ago

Hello @Makavelic,

Please clarify whether you used the following command to run tests:

testcafe "path:d:\edgeportable\msedge.exe --headless"  test.js

In your examples, the --headless and --headless=new options are outside the quotation marks. Therefore, they are parsed as TestCafe options rather than browser options.

Sorry about that, I've edited my post with the correct syntax. It still has the same errors.

Bayheck commented 1 month ago

Hello,

Could you please share the error message you receive?

Makavelic commented 1 month ago

Like I mentioned, the error is coming from edge itself not testcafe:

The application was unable to start correctly (0x0000022) Click OK to close the application. The same error happens when running it directly without testcafe.

The only way to get it to run properly without testcafe is to run with --headless=new. However, I have been unsuccessful forcing headless=new in testcafe for edge.

Bayheck commented 1 month ago

Hello, try running your tests as follows:

npx testcafe "edge:d:\edgeportable\msedge.exe --headless=new" test.js

or

npx testcafe "edge:d:\edgeportable\msedge.exe --headless" test.js

Note: to launch a custom headless browser, replace the path: prefix with the browser’s alias.

https://testcafe.io/documentation/402828/guides/intermediate-guides/browsers#test-in-headless-mode

Makavelic commented 1 month ago

I get the following error when I try your example:

ERROR Cannot establish one or more browser connections. 1 of 1 browser connections have not been established:

This occurs whether I use the edge: or path: syntax.

Bayheck commented 4 weeks ago

I was not able to reproduce this behavior with the local version of edge.

Perhaps the issue only occurs in your portable edge. Could you please share it with us?

Could you also execute your tests in the same manner in a different browser (such as Chrome)?

To help you further, we need a repository with an example that shows how you execute your tests (a simple sample is sufficient). Please share all the paths and script commands in the config. This way, we will be in a better position to assist you.

github-actions[bot] commented 2 weeks ago

This issue was automatically closed because there was no response to our request for more information from the original author. Currently, we don't have enough information to take action. Please reach out to us if you find the necessary information and are able to share it. We are also eager to know if you resolved the issue on your own and can share your findings with everyone.

Makavelic commented 3 days ago

Was able to find a workaround by adding --no-sandbox. Not sure why this is needed on newer versions of edge.