Test hangs when using Role

secilkaplan commented 1 year ago

What is your Scenario?

Our test project uses TestCafe together with gherkin-testcafe.

We have 11 groups of test jobs and each has multiple feature files and multiple scenarios in it. Mostly the feature files use a single email to login to application and we have a login step in the gherkin Background of the feature files.

We use a single Role which takes email as parameter and in each scenarios before hook we login with that email. Some scenarios also login to another user during test execution.

What is the Current behavior?

After upgrading TestCafe from 1.20.1 to the latest version we started to experience a test hang issue during login using Role. We have also tried 3.2.0 and 2.6.0 versions.

The problem is not always reproducible. But sometimes happens when tests are started locally or on CI (CircleCI). Sometimes the problem happens in the first scenario in the test job and sometimes after running some successful scenarios. Because the CI job has a no_output_timeout of 20 m we see the console log 'Too long with no output (exceeded 20m0s): context deadline exceeded'

When the problem is reproducible (as we are running the test locally) here is what we see:

the login page is opened,
email is entered into the field
test hangs (Does not fail it but also does nothing to continue the steps to enter data into the password field)

We have also tried putting some console logs to debug to understand where it hangs. On CI run we have observed it also failed even when entering email. So we are mostly sure it always happens inside the Role.

Looks very similar to an old issue which was fixed here https://github.com/DevExpress/testcafe/issues/5278

What is the Expected behavior?

When there is an exception or error test should fail and not hang

What is the public URL of the test page? (attach your complete example)

Even though I have prepared an example with TestCafe & gherkin-testcafe I'm not able to reproduce this problem on the example so I haven't shared here.

What is your TestCafe test code?

Here is the Role description.

export const jm_user = (email) =>
  Role(
    loginPage.url,
    async (t) => {
      await t
        .typeText(loginPage.username, email, {
          replace: true,
          paste: true,
        })
        .typeText(loginPage.password, process.env.USER_PASSWORD, {
          replace: true,
          paste: true,
        })
        .click(loginPage.loginButton)
    },
    { preserveUrl: true }
  )

Here is the gherkin login step implementation where we call the Role. user is passed with the gherkin step parameter.

Given(/^user is logged in with email "(.+)"$/, async (t, [user]) => {
    await t.useRole(jm_user(user))
})

Your complete configuration file

{
  "browsers": "chrome:headless",
  "src": ["tests/**/*"],
  "hostname": "localhost",
  "screenshots": {
    "path": "tests/screenshots/",
    "takeOnFails": true,
    "pathPattern": "${FIXTURE}-${TEST}.png"
  },
  "reporter": [
    { "name": "gtc-reporter-spec" },
    { "name": "xunit", "output": "testcafe-report.xml" },
    {
      "name": "cucumber-json-gherkin",
      "output": "cucumber-json-reports/cucumber-report.json"
    }
  ],
  "quarantineMode": {
    "successThreshold": 1,
    "attemptLimit": 3
  },
  "stopOnFirstFail": false,
  "skipJsErrors": true,
  "skipUncaughtErrors": true,
  "concurrency": 1,
  "selectorTimeout": 3000,
  "assertionTimeout": 3000,
  "pageLoadTimeout": 6000,
  "ajaxRequestTimeout": 30000,
  "pageRequestTimeout": 30000,
  "disablePageCaching": true,
  "disableMultipleWindows": false,
  "color": true,
  "disableNativeAutomation": "true"
}

Your complete test report

No response

Screenshots

No response

Steps to Reproduce

1. 2. 3.

TestCafe version

3.3.0

Node.js version

18.17

Command-line arguments

-

Browser name(s) and version(s)

chrome 118, chrome 119

Platform(s) and version(s)

macOs Sonoma, Linux on CI using image cimg/node:18.17-browsers

Other

The problem does not happen with old version 1.20.1 So far what we have tried and still did not work to overcome the hang issue is as below

removing paste:true, replace:true in typeText
adding a doubleClick before typeText (thinking maybe focusing was the problem)
removing the email parameter from Role and passing it hardcoded
defining another login function in our helper class without using Role (but we still used useRole(anonymous()) to logout before logging for the next scenario)

AlexKamaev commented 1 year ago

It's difficult to determine the cause of the issue without a sample project. I see that you already made several attempts to make the tests work.

Since your tests still do not work even after you removed Roles, it's unlikely that the issue is role-related. I can only assume that TestCafe ignores the next command after login. You can try to add an empty command between the steps to check this assumption. For example: await t.eval(() => {}).

I also noticed that you enable the disableNativeAutomation option. Is it because you need to open new windows during the login procedure? If not, please remove this option and run tests in Native Automation mode.

I also noticed that you enable the skipJsErrors option. This can lead to hanging tests. Please check the browser console after the test hangs. If it has errors unrelated to your code, please share them with us.

Please try to reproduce the issue without testcafe-gherkin to make sure that it's a TestCafe issue and not related to the testcafe-gherkin module.

If this does not help, we will need an example to research the issue further.

secilkaplan commented 1 year ago

Thanks @AlexKamaev , I'll try all suggestions and get back if I can come up with a reproducible example.

github-actions[bot] commented 11 months ago

This issue was automatically closed because there was no response to our request for more information from the original author. Currently, we don't have enough information to take action. Please reach out to us if you find the necessary information and are able to share it. We are also eager to know if you resolved the issue on your own and can share your findings with everyone.

DevExpress / testcafe