DevExpress / testcafe

A Node.js tool to automate end-to-end web testing.
https://testcafe.io
MIT License
9.83k stars 673 forks source link

Execution hangs after test fail on chrome version 130 #8307

Closed cheljo closed 1 month ago

cheljo commented 1 month ago

What is your Scenario?

After chrome update on version 130, when I start test execution I see that test is failing but instead of displaying error log it just stops. If I restart my computer, sometimes it works, but in 99% of executions it doesn't. I tried with visual studio code and intellij, I face same issue. I tried firefox and edge and did not have this problem. I also downgraded chrome to version 103 and it works well. This only happens on chrome version 130. Can you hepl me please?

What is the Current behavior?

After test fail, error log is not displayed and execution is not finished and it just hangs until I stop it.

What is the Expected behavior?

After test fail, error log should be displayed and execution should be finished.

What is the public URL of the test page? (attach your complete example)

Every tc.

What is your TestCafe test code?

Fail some test.

Your complete configuration file

No response

Your complete test report

No response

Screenshots

No response

Steps to Reproduce

  1. Start tc where fail is expected
  2. After fail of test, execution hangs

TestCafe version

3.6.0

Node.js version

20.11.0

Command-line arguments

testcafe chrome src/test_suites/*-tests.ts --page-load-timeout 10000 --assertion-timeout 15000 --selector-timeout 20000 --debug-on-fail --skip-js-errors --disable-native-automation --reporter html:reports/htmlReports/TestReports.html --screenshots path=reports/ss,takeOnFails=true

Browser name(s) and version(s)

Chrome 130

Platform(s) and version(s)

Windows

Other

No response

gforepsly commented 1 month ago

I can confirm we are experiencing the same issue.

For example: test('Test block 1', async t => { await t.expect('1').eql('2'); }); test('Test block 2', async t => { console.log('Here'); });`

Test block 2 is never reached, the browser just hangs using newest Chrome (I use MAC if that makes a difference, on latest 15 macOS.

We use spec reporter using .testcaferc.json file: { "browsers": ["chrome"], "src": "testFile", "reporter": [ { "name": "spec" } ], "skipJsErrors": true, "skipUncaughtErrors": true, "hostname": "localhost", "screenshots": { "path": "testcafe/results/screenshots/", "takeOnFails": true, "pathPattern": "${DATE}_${TIME}/${USERAGENT}/${FIXTURE}/${TEST_INDEX}_${TIME}.png", "fullPage": true, "thumbnails": false } }

The problem seems to be in the spec reporter, it works using json report, but that doesn't really show the error in the console. That is really bad for local development and such.

cheljo commented 1 month ago

OK, thank you and please keep us updated.

gforepsly commented 1 month ago

OK, thank you and please keep us updated.

I'am not part of the testcafe development team, so I also need a reply from them on this :) But it seems fairly straightforward to reproduce. Can you also try without using the spec reporter and let us know if that is the case for you as well?

cheljo commented 1 month ago

Hey @gforepsly yes, you are right. When I disable spec reporter I don't have problems.

gforepsly commented 1 month ago

Hey @gforepsly yes, you are right. When I disable spec reporter I don't have problems.

Thx for confirming! Now we wait for DEVs to respond :)

Suchiraz commented 1 month ago

How do we remove the spec reporter from testcafe?

Suchiraz commented 1 month ago

Hey @gforepsly yes, you are right. When I disable spec reporter I don't have problems.

How did u do this?

gforepsly commented 1 month ago

How do we remove the spec reporter from testcafe?

If you use the configuration file, remove "spec" entry from the reporter array in the config file. If you call the reporter via CLI, you can just use "json" as a reporter, without "spec" reporter. You can refer to the official documentation to check all of the options.

Spec reporter is great as it shows you the errors in the console in real time - we use it heavily in our CI tool so we are blocked with this bug.

Bayheck commented 1 month ago

Hello,

We have released testcafe 3.7.0-rc.1 that addresses this issue.

I will close this issue as a duplicate of https://github.com/DevExpress/testcafe/issues/8300.

However, you might face a new issue, which we described here: https://github.com/DevExpress/testcafe/issues/8308

Please take a look and let us know your results.