DevExpress / testcafe

A Node.js tool to automate end-to-end web testing.
https://testcafe.io
MIT License
9.83k stars 673 forks source link

Test failures getting stuck and not generating test report #8324

Closed usmanm16 closed 3 weeks ago

usmanm16 commented 3 weeks ago

What is your Scenario?

I use this command testcafe -c 5 firefox:headless to run regression suite. I have about 200 test cases in regression suite. I am using "testcafe": "^3.6.2", "testcafe-blink-diff": "0.4.11", "testcafe-reporter-allure-expanded": "1.1.0", "testcafe-reporter-html": "^1.4.6",

I have tried both Firefox and and Chrome, same issue with both browsers.

What is the Current behavior?

Whenever i run regression suite, if any one of the test fails. It gets stuck, do not end execution after completing the job and do not generate test report. Because test report doesn't generate until execution complete.

What is the Expected behavior?

It should completed execution, it should take screenshot of all tests failed and generate test report.

What is the public URL of the test page? (attach your complete example)

https://www.clinicalkey.com/student/login

What is your TestCafe test code?

Any simple login and logout code. Same behaviour.

Your complete configuration file

{ "src": "testcafe/tests", "skipJsErrors": true, "disableNativeAutomation": "true", "screenshots": { "path": "testcafe/screenshots", "pathPattern": "$${DATE}_${TIME}/${TESTID}/${TEST}.png", "takeOnFails": true, "fullPage": true }, "reporter": { "name": "html", "output": "testcafe/reports/htmlReports/POMReports.html" } }

Your complete test report

Can't generate test report

Screenshots

In the screenshot test-1 is failed and doesn't display screenshot of failing page. In the screenshot test-7 is failed but do display screenshot of failing page. Screenshot 2024-10-29 at 11 56 52 AM

Steps to Reproduce

  1. Run regression suite headless mode
  2. If any test fails
  3. If i am using 5 concurrence, one with failure get stuck and i am down to 4. If one more test fails then i am down to 3 tabs.

TestCafe version

3.6.2

Node.js version

v21.6.1

Command-line arguments

testcafe -c 5 firefox:headless ./testcafe/tests/.test/.test.js --hostname localhost

Browser name(s) and version(s)

Chrome is up to date Version 130.0.6723.70 (Official Build) (x86_64). FireFox: Version 128.3.1esr (64-bit)

Platform(s) and version(s)

macOS 13.7 (22H123)

Other

It would be great if someone can help me debug this issue

alex-key commented 3 weeks ago

I have the same on Windows. Tab is just stuck, no debug panel visible in the browser and no report in console provided.

Bayheck commented 3 weeks ago

Hello, the issue was fixed in https://github.com/DevExpress/testcafe/issues/8300.

We are working on the TestCafe release with the fix, but an ETA could be misleading.

In the meantime, you can try using TestCafe 3.7.0-rc.2 as a temporary solution.

I will close this issue.

alex-key commented 2 weeks ago

Thank you, it works well after update to v3.7.0