cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.54k stars 3.2k forks source link

launchoptions args (for incognito) are only being passed for the first spec and are not present in the 2nd spec in run mode #29782

Open xyuste opened 5 months ago

xyuste commented 5 months ago

Current behavior

Hi, I have one test project with 2 features (I am using cucumber preprocessor @badeball), then when I try to execute my test with: npx cypress run --browser edge --headed

Only the first feature is executed in the "inprivate" mode, the second one is executed in normal mode.

Desired behavior

All the features should be executed inprivate mode I have created one ticket in the @badeball support, and the ticket has been closed with : I doubt this pertains to the preprocessor, hence closing.

Also it happens executing one project with several tests (including the "2-advanced-examples", downloaded from the cypress web), and starting from command line: npx cypress run --browser edge --headed only the first test is executed inprivate mode all the rest in normal mode

Where is the problem?

Many thanks!

Test code to reproduce

My file tsconfig.json:

const { defineConfig } = require("cypress");
import createBundler from "@bahmutov/cypress-esbuild-preprocessor";
import { addCucumberPreprocessorPlugin } from "@badeball/cypress-cucumber-preprocessor";
import createEsbuildPlugin from "@badeball/cypress-cucumber-preprocessor/esbuild";

module.exports = defineConfig({
  e2e: {
    specPattern: "**/*.feature",
    async setupNodeEvents(
      on: Cypress.PluginEvents,
      config: Cypress.PluginConfigOptions
    ): Promise<Cypress.PluginConfigOptions> {
      await addCucumberPreprocessorPlugin(on, config);
      on(
        "file:preprocessor",
        createBundler({
          plugins: [createEsbuildPlugin(config)],
        })
      );
      on('before:browser:launch', (browser = {}, launchOptions) => {
        // `args` is an array of all the arguments that will
        // be passed to browsers when it launches
        console.log(launchOptions.args) // print all current args
        if (browser.name === 'edge') {
          // open in incognito
          launchOptions.args.push('--inprivate')
        }
        if (browser.family === 'chromium' && browser.name !== 'electron') {
          launchOptions.args.push("--incognito");                
        }
          // whatever you return here becomes the launchOptions
        return launchOptions
      })
      return config;
    },
  },
  experimentalModifyObstructiveThirdPartyCode: true,
  chromeWebSecurity: true,
});

Cypress Version

13.12.0

Node version

v20.13.0

Operating System

windows 10

Debug Logs

The log with the error, caused because the execution is not using the inprivate mode:

    AssertionError: Timed out retrying after 4000ms: Expected to find element: `input[type="email"]`, but never found it.
      at navigate.eval [as enterCorrectCredentials] (cypress/integration/navigate.js:21:7)
      at Context.eval (cypress/integration/uefana.js:12:11)
      at Context.resolveAndRunStepDefinition (node_modules/cypress-cucumber-preprocessor/lib/resolveStepDefinition.js:193:0)
      at Context.eval (node_modules/cypress-cucumber-preprocessor/lib/createTestFromScenario.js:27:0)
      at getRet (************************************************/__cypress/runner/cypress_runner.js:118799:20)
      at tryCatcher (************************************************/__cypress/runner/cypress_runner.js:1807:23)
      at Promise.attempt.Promise.try (https://na-data-collection.dev.enterprise.uefa.com/__cypress/runner/cypress_runner.js:4315:29)
      at Context.thenFn (************************************************/__cypress/runner/cypress_runner.js:118810:66)
      at Context.then (************************************************/__cypress/runner/cypress_runner.js:119061:21)

Other

I am using following dependencies: "dependencies": { "@badeball/cypress-cucumber-preprocessor": "^20.1.0", "@bahmutov/cypress-esbuild-preprocessor": "^2.2.1", "cypress-xpath": "^2.0.1"

jennifer-shehane commented 5 months ago

@xyuste Is it the first specfile that executes in inprivate mode? Or is it only the first test that executes in inprivate mode. I haven't seen someone use this flag previously, so I don't have validation on how this works. It seems odd that it works once though.

xyuste commented 5 months ago

Hi Jennifer, We have detected in a project using cucumber @badeball preprocessor, but now, I have created a more easy project without the Cucumber preprocessor, and it happens the same. I will try to answer your questions:

  1. The first specfile is executed in the inprivate mode using Edge, or incognito mode using Chrome. In the below example the first spec (example_test.cy.js) has 2 "it" which are executed in the inprivate/incognito modes correctly. But the second spec (login_get.cy.js) is executed in the normal mode and for this reason it causes error: Spec Tests Passing Failing Pending Skipped │ √ example_test.cy.js 00:15 2 2 - - - │ │ × login_get.cy.js 00:21 1 - 1 - - │

  2. In Edge the incognito mode, is called inprivate.

  3. This is the cypress.config.js file in the second project:

const { defineConfig } = require("cypress");

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      on('before:browser:launch', (browser = {}, launchOptions) => {
        // `args` is an array of all the arguments that will
        // be passed to browsers when it launches
        console.log(launchOptions.args) // print all current args

        if (browser.name === 'edge') {
          // open in incognito
          launchOptions.args.push('--inprivate')
        }

        if (browser.family === 'chromium' && browser.name !== 'electron') {
          launchOptions.args.push("--incognito");                
        }

        // whatever you return here becomes the launchOptions
        return launchOptions
      })      
    },

    experimentalModifyObstructiveThirdPartyCode: true,
    chromeWebSecurity: true
  },
});

And this the command used to launch the test: npx cypress run --browser chrome --headed Many thanks and regards Xavier

jennifer-shehane commented 5 months ago

I'm able to reproduce this in Chrome with the instructions below. This seems really problematic if this is how Cypress is behaving.

npm i detectincognitojs --save

cypress.config.js

const { defineConfig } = require("cypress");

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      on('before:browser:launch', (browser = {}, launchOptions) => {
        if (browser.family === 'chromium' && browser.name !== 'electron') {
          launchOptions.args.push("--incognito");                
        }

        return launchOptions
      })      
    },
  },
});

test.cy.js

import { detectIncognito } from "detectincognitojs";

it('test 1', (done) => {
  detectIncognito().then((result) => {
    expect(result.browserName).to.equal('Chrome');
    expect(result.isPrivate).to.be.true;
    done()
  });
});

test2.cy.js

import { detectIncognito } from "detectincognitojs";

it('test 2', (done) => {
  detectIncognito().then((result) => {
    expect(result.browserName).to.equal('Chrome');
    expect(result.isPrivate).to.be.true;
    done()
  });
});
cypress run --browser chrome

Screenshot 2024-07-02 at 12 59 26 PM

jennifer-shehane commented 5 months ago

I tracked this down to being introduced in Cypress 12.15.0 which is when we began passing the --headless=new flag through to Chromium browsers. https://github.com/cypress-io/cypress/pull/26481 I suspect this is the reason for this issue somehow.

jennifer-shehane commented 5 months ago

Ok, I've narrowed down some specifics some more.

This behavior has actually been happening in --headed mode since 10.0.0 (I did not try earlier versions).

When we switched the --headless=new flag in 12.15.0, that introduced this behavior to headless mode as well (since it more closely matches the implementation of the headed browser.

This expansion to --headless having this issue was introduced in this commit: https://github.com/cypress-io/cypress/commit/5b27edd6f29abb72610c0c8531d1584353ac1695

I'm unsure at this point if this issue is isolated to --incognito flag specifically or all flags passed through to Chrome.

luiseps commented 4 months ago

@jennifer-shehane I'm facing same issue with Edge and Chrome browser.

andresrvr02cr commented 4 months ago

I'm facing this issue too :( this is useful for AAD authentication for avoid to take current sessions.

luiseps commented 3 months ago

@jennifer-shehane is there any update about this issue?... Seems it started in the latest chrome and Edge versions (Chromium based browsers) not only with the 12.5 Cypress version.