search

DevExpress / testcafe

A Node.js tool to automate end-to-end web testing.
https://testcafe.io
MIT License
9.82k stars 670 forks source link

chrome:headless (MacOS) results with ' 1) AssertionError: expected 'about:blank' to include $target page' #6948

Closed Gjodim closed 1 year ago

Gjodim commented 2 years ago

What is your Scenario?

0

I am using TestCafe in combination with gherkinTestcafe (steps) / cucumber. I am also using environment variables so that i can run my tests on 2 different environments.

My code is as follows, although through debugging, i don't believe this is something strictly code related, as much as it is related to:

import Enviorments from "../../../../../../AEM_Engine/Enviorment/Enviorments";
import { Helper } from "../../../../../TestActions/Test_specific/Career_helper";
import {AddAuthCredentialsHook} from "../../../../../TestActions/BasicAuth";

const {Before, Given, Then} = require('cucumber');
let publisher = new Publish();
let aemEnv = new Enviorments();
let helper = new Helper;

let careersPage = '/career';

Before('@basicAuth', async testController => {
    const addAuthCredentialsHook = new AddAuthCredentialsHook('$someUserName', '$somePassword');
    await testController.addRequestHooks(addAuthCredentialsHook);
});

Before('@disableCookie', async testController => {
    await testController.addRequestHooks(publisher.mockCookieResponse);
});

Given('I am at Careers page', async testController => {
    await publisher.Navigate(testController, aemEnv.frontEndURL + careersPage);
    await publisher.verifyURL(testController, aemEnv.frontEndURL + careersPage);
});

.
.
.

When i wait for the script to run i have

1) AssertionError: expected 'about:blank' to include $expectedPage As i mentioned, i don't believe the problem is in the code. Even if i remove the step for verifying the current URL location, the test fails on the next step after. Tests pass on

Chrome (with UI shell) Other browsers (firefox, safari), headless or with UI shell Second (staging) environment When Tests are run and TestCafe starts, i get the following info

Running tests in:

I tried re-installing some packages, re-writing some of the steps, adding some flags to clear cache, change chrome port or similar, but nothing worked.

Any thoughts on what might be causing this and how to solve it?

What is the Current behavior?

Tests do not run in chrome:headless (see error above in the description)

What is the Expected behavior?

Tests should run in chrome:headless.

What is your public website URL? (or attach your complete example)

cannot disclose, can try if it's crucial.

What is your TestCafe test code?

import Enviorments from "../../../../../../AEM_Engine/Enviorment/Enviorments"; import { Helper } from "../../../../../TestActions/Test_specific/Career_helper"; import {AddAuthCredentialsHook} from "../../../../../TestActions/BasicAuth";

const {Before, Given, Then} = require('cucumber'); let publisher = new Publish(); let aemEnv = new Enviorments(); let helper = new Helper;

let careersPage = '/career';

Before('@basicAuth', async testController => { const addAuthCredentialsHook = new AddAuthCredentialsHook('$someUserName', '$somePassword'); await testController.addRequestHooks(addAuthCredentialsHook); });

Before('@disableCookie', async testController => { await testController.addRequestHooks(publisher.mockCookieResponse); });

Given('I am at Careers page', async testController => { await publisher.Navigate(testController, aemEnv.frontEndURL + careersPage); await publisher.verifyURL(testController, aemEnv.frontEndURL + careersPage); });

. . .

Your complete configuration file

No response

Your complete test report

No response

Screenshots

No response

Steps to Reproduce

0

I am using TestCafe in combination with gherkinTestcafe (steps) / cucumber. I am also using environment variables so that i can run my tests on 2 different environments.

My code is as follows, although through debugging, i don't believe this is something strictly code related, as much as it is related to:

import Enviorments from "../../../../../../AEM_Engine/Enviorment/Enviorments";
import { Helper } from "../../../../../TestActions/Test_specific/Career_helper";
import {AddAuthCredentialsHook} from "../../../../../TestActions/BasicAuth";

const {Before, Given, Then} = require('cucumber');
let publisher = new Publish();
let aemEnv = new Enviorments();
let helper = new Helper;

let careersPage = '/career';

Before('@basicAuth', async testController => {
    const addAuthCredentialsHook = new AddAuthCredentialsHook('$someUserName', '$somePassword');
    await testController.addRequestHooks(addAuthCredentialsHook);
});

Before('@disableCookie', async testController => {
    await testController.addRequestHooks(publisher.mockCookieResponse);
});

Given('I am at Careers page', async testController => {
    await publisher.Navigate(testController, aemEnv.frontEndURL + careersPage);
    await publisher.verifyURL(testController, aemEnv.frontEndURL + careersPage);
});

.
.
.

When i wait for the script to run i have

1) AssertionError: expected 'about:blank' to include $expectedPage As i mentioned, i don't believe the problem is in the code. Even if i remove the step for verifying the current URL location, the test fails on the next step after. Tests pass on

Chrome (with UI shell) Other browsers (firefox, safari), headless or with UI shell Second (staging) environment When Tests are run and TestCafe starts, i get the following info

Running tests in:

I tried re-installing some packages, re-writing some of the steps, adding some flags to clear cache, change chrome port or similar, but nothing worked.

Any thoughts on what might be causing this and how to solve it?

TestCafe version

1.3.3

Node.js version

12.18.1

Command-line arguments

'chrome:headless --start-fullscreen' "Automated_Tests/TestSuite/Regression/features/step_definitions/Career/.js" "Automated_Tests/TestSuite/Regression/features/Career/.feature" --ENV=https://www.testPage.com --skip-js-errors --page-load-timeout 6000 --selector-timeout 6000 --assertion-timeout 6000 --private-mode -e --no-user-gesture-requires

Browser name(s) and version(s)

HeadlessChrome 99.0.4844

Platform(s) and version(s)

Mac OS X 10.15.7

Other

No response

VasilyStrelyaev commented 2 years ago

Thank you for your report. Unfortunately, we cannot investigate this issue without a complete example that we could run on our side. This example should contain all imported files and include the tested application's URL.

If you cannot disclose the test code and URL publicly, you can mail them to our Support Team at support@devexpress.com.

Gjodim commented 2 years ago

I sent an email with the information to support@devexpress.com. Please contact if you need something else.

VasilyStrelyaev commented 2 years ago

Unfortunately, I am not able to find your email in our mailbox. Could you please re-send your message and specify a part of your email address, or the message subject, so that we could use search to find it.

Gjodim commented 2 years ago

I just resend it @VasilyStrelyaev starts with dimov_g

VasilyStrelyaev commented 2 years ago

Unfortunately, the sample you sent seems to be incomplete. The test file imports the Publish.js file, which, in turn, imports other files like AEM_WebEngine.js, FE_Engine.js, etc. The latter files cannot be found in your message's attachments. That is why I cannot run the test you shared and reproduce the issue.

You can prepare a portable project more quickly and reduce the number of turnarounds if you do the following: 1) create a blank project on your local machine 2) add package.json and the sample test file (for instance, 03_SearchFiltersSection.js) to this project 3) copy any referenced files to the project's directory (or comment out references to these files, if they are not required to run the sample test) and add the required modules to package.json 4) ensure that you can run the sample test in this stand-alone project after you added all the necessary files and module references 5) remove the node_modules folder, archive the project, and send it to support@devexpress.com

Gjodim commented 2 years ago

Hello @VasilyStrelyaev . Yes, i am aware of that. These references had nothing to do with the code, but i see your point. I cannot do much unfortunately, since any further references are beyond my knowledge and can add to the confusion without providing much value. Also, many of these contain information which i cannot share straight forward which further adds to complicating things.

I did some changes though to the test file 03_SearchFiltersSection, so that it doesn't include any unnecessary references (publish.js). Please tell me if this works for You as well. I tested again on chrome v chrome:headless and i get the same result - works on chrome but not on chrome:headless. Also again - works overall on stage environment but not on production (chrome:headless)

Artem-Babich commented 2 years ago

Hi, I assume that this issue can be related to multiple factors and combination of your libraries. However, I'm unable to create a working example based on the code files you shared. I'm afraid we cannot determine the cause of the issue without being able to reproduce it locally. That is why I ask you to create a runnable sample project that does not contain confidential information and illustrates the issue: How To: Create a Minimal Working Example When You Submit an Issue

Gjodim commented 2 years ago

Hello. @Artem-Babich @VasilyStrelyaev

I created a simple project using code from the real one. The result is the same. I will send you a compressed version of it as well as an explanation on the mail support@devexpress.com

AlexKamaev commented 2 years ago

@Gjodim Thank you for your cooperation. I managed to reproduce the issue. I confirm that the issue only exists when chrome is launched in the headless mode. Also, it seems that the issue is not specific to MacOS only. I was able to reproduce it on Windows as well. Please track this thread for updates.

Gjodim commented 2 years ago

Hello @AlexKamaev @VasilyStrelyaev Any progress on this one?

github-actions[bot] commented 2 years ago

No updates yet. Once we get any results, we will post them in this thread.

Artem-Babich commented 1 year ago

Hello @Gjodim

We've released a new TestCafe version v2.3.0 that includes an experimental proxyless mode. This mode uses native browser automation. In Proxyless mode, a few issues are already fixed. This issue should also be fixed in Proxyless mode.  I just tried to reproduce the issue using your example, but it looks like your website has changed. Also, I tried to run the test in headless mode and got the Access Denied error. It looks like your website checks if the browser is running in headless mode and restricts access. Would you please check if your sample is working correctly in v2.3.0 with the experimental-proxyless mode enabled? 

This option is available in all interfaces:

// Command-line
testcafe chrome tests --experimental-proxyless 

// Programmatic
const testcafe = await createTestCafe({ experimentalProxyless: true });

// Configuration file
{
   "experimentalProxyless": "true"
}    

Note that at present it is an experimental mode. Also, the Proxyless mode is implemented only in Google Chrome. It will not work correctly if you run tests in a non-Chrome browser or in a combination of other browsers.

github-actions[bot] commented 1 year ago

This issue was automatically closed because there was no response to our request for more information from the original author. Currently, we don't have enough information to take action. Please reach out to us if you find the necessary information and are able to share it. We are also eager to know if you resolved the issue on your own and can share your findings with everyone.