[BUG] chrome opens with cookies present

[karpikpl]

System info

• Playwright Version: [v1.34]
• Operating System: [Windows 11]
• Browser: [Chromium]
• Other info: Using Azure AD (Entra), MSAL library for Angular

Source code

• [X] I provided exact source code that allows reproducing the issue locally. (this issue is hard to reproduce as it requires an Azure AD protected website and AD joined VM)

Config file

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: [
    ['html'],
    ['line'],
    ['junit', { outputFile: './test-results/results.xml' }],
    ['playwright-trx-reporter', { outputFile: "./test-results/output.trx" }]
  ],
  use: {
    trace: 'retain-on-failure',
  },
  projects: [
    { name: 'setup', testMatch: /.*\.setup\.js/ },
    {
      name: 'chromium',
      testMatch: /.*aad\.spec\.js/,
      use: { 
        ...devices['Desktop Chrome'],
       storageState: 'playwright/.auth/user.json',
      },
     dependencies: ['setup'],
    },
  ],
});

Test file (self-contained)

import { test as setup } from '@playwright/test';

const authFile = 'playwright/.auth/user.json';
setup.use({storageState: {cookies: [], origins: []}});

setup('authenticate', async ({ context }, testInfo) => {

  await context.clearCookies();
  var page = await context.newPage();

  console.log('Using url %s', process.env.TEST_URL);
  await page.goto(process.env.TEST_URL);

  var state = await context.storageState();
  console.dir(state);

  await page.waitForURL((url) => url.host === 'login.microsoftonline.com');
  state = await context.storageState();
  console.dir(state);

  console.log('Using username %s', process.env.TEST_USERNAME);

  try {
    // this is where SSO kick in and automatically logs the user
    await page.getByRole('link', { name: 'Cancel' }).click();
  }
  catch
  {
    // SSO kicked in?
    console.log('Failed to click cancel SSO');
  }

  await page.getByPlaceholder('UserID@XXX.com').fill(process.env.TEST_USERNAME, {timeout: 5000});
  await page.getByRole('button', { name: 'Next' }).click();

  await page.getByPlaceholder('Password').fill(process.env.TEST_PASSWORD);
  await page.getByRole('button', { name: 'Sign in' }).click();

  // Do not stay signed in
  await page.getByRole('button', { name: 'No' }).click();
  // Use try catch block to handle the case where the consent dialog is shown for the first login
  // try {
  //     await page.waitForURL('**/Consent/**');
  //     await page.getByRole('button', { name: 'Yes' }).click();
  // } catch (e) {
  // }
  // Save auth state to file (.gitignore'd)

  // Wait until page has changed and is loaded
  await page.waitForURL(process.env.TEST_URL)

  await context.storageState({ path: authFile });
});

Steps

• [Run the test]

Expected

This test runs on Azure AD joined Windows 11. I expected that every time setup runs - it gets clear browser context without cookies and AD login screen with user/password is available.

Actual

Test gest to TEST_URL which redirects to Azure AD login screen, but then user gets automatically logged in. When I list cookies after redirect to login.microsoftonline.com, there are present - but playwright should open an incognito window?

I tried:

import { test as setup } from '@playwright/test';

const authFile = 'playwright/.auth/user.json';
setup.use({storageState: {cookies: [], origins: []}});

setup('authenticate', async ({ context }, testInfo) => {

but console log shows cookies present (after redirect to login page):

await context.clearCookies();
  var page = await context.newPage();

  console.log('Using url %s', process.env.TEST_URL);
  await page.goto(process.env.TEST_URL);

  var state = await context.storageState();
  console.dir(state); // Application insights cookie only

  await page.waitForURL((url) => url.host === 'login.microsoftonline.com');
  state = await context.storageState();

Am I missing something here? Same test works OK in CI system but fails locally. I can also reproduce it using npx playwright codegen - it gets me logged in automatically. Opening a new incognito window from codegen chromium works fine. Opening an incognito window outside of playwright also works.

[RAMKUMARSHUKLA]

Try clearing cookies using await context.clearCookies(); and creating new context within browser. Following is the updated code for Test file.

import { test as setup } from '@playwright/test';

const authFile = 'playwright/.auth/user.json';

setup('authenticate', async ({ context }, testInfo) => { // Clear cookies before starting the test await context.clearCookies();

var page = await context.newPage();

console.log('Using url %s', process.env.TEST_URL); await page.goto(process.env.TEST_URL);

var state = await context.storageState(); console.dir(state); // Application insights cookie only

await page.waitForURL((url) => url.host === 'login.microsoftonline.com'); state = await context.storageState();

// Perform authentication steps. Replace these actions with your own. console.log('Using username %s', process.env.TEST_USERNAME);

await page.getByPlaceholder('UserID@XXX.com').fill(process.env.TEST_USERNAME); await page.getByRole('button', { name: 'Next' }).click(); await page.getByPlaceholder('Password').fill(process.env.TEST_PASSWORD); await page.getByRole('button', { name: 'Sign in' }).click();

// Do not stay signed in await page.getByRole('button', { name: 'No' }).click();

// Wait until page has changed and is loaded await page.waitForURL(process.env.TEST_URL)

await page.context().storageState({ path: authFile }); });

[karpikpl]

I actually tried it but failed to include it in my source code (but you can see it in the bottom sample). I've updated the sample with the latest code I tried.

// Clear cookies before starting the test
await context.clearCookies();

I searched high and low but I cannot find a definitive answer on how "clean" chromium instance is supposed to be. Playwrights documentation says that each context is independent, shouldn't that mean that no cookies are included?

I feel like I'm missing something :/

[mxschmitt]

Just to confirm, as per your configuration, you are not using Google Chrome, and are using just Chromium, is that correct?

Your reproducible is not working, would it be possible to share a working reproducible with us? Feel also free to ping me internally on Teams, ideally with test-credentials.

Could you try putting the following inside your playwright.config.ts file?

    {
      name: 'chromium',
      testMatch: /.*aad\.spec\.js/,
      use: { 
        ...devices['Desktop Chrome'],
       storageState: 'playwright/.auth/user.json',
       launchOptions: {
        args: ['--auth-server-allowlist="*"']
      }
      },
     dependencies: ['setup'],
    },

[karpikpl]

Hi, thanks for getting back to me. I'm still facing this issue. It worked for me once, but then when I run npx playwright test I see in trace logs that Azure AD automatically logs me in and there's no .auth saved session data.

The way I understand playwright, is that this should never happen, because the browser should open in incognito mode with no state (cookies).

[karpikpl]

I also tried with updated playwright.ts file. First I got in powershell:

Error: Playwright Test did not expect test() to be called here.
Most common reasons include:
- You are calling test() in a configuration file.
- You are calling test() in a file that is imported by the configuration file.
- You have two different versions of @playwright/test. This usually happens
  when one of the dependencies in your package.json depends on @playwright/test.

my package.json is the tiny:

{
  "name": "ui-tests",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {},
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@playwright/test": "^1.34.3",
    "dotenv": "^16.3.1"
  }
}

Then I tried in cmd and got the same error as previously - test got automatically logged in so it couldn't find the login box

[karpikpl]

After trying more things - this seemed to fix it, but I don't know why

    {
      name: 'setup', testMatch: /.*\.setup\.js/,
      use: {
        ...devices['Desktop Chrome'],
        launchOptions: {
          args: ['--auth-server-allowlist="*"']
        }
      }
    },

    {
      name: 'chromium',
      testMatch: /.*aad\.spec\.js/,
      use: {
        ...devices['Desktop Chrome'],
        storageState: 'playwright/.auth/user.json',
        launchOptions: {
          args: ['--auth-server-allowlist="*"']
        }
      },
      dependencies: ['setup'],
    },

[mxschmitt]

Folding into https://github.com/microsoft/playwright/issues/17328.

[karpikpl]

I think that's a different issue. In my case I'm pretty sure it's the Entra Seamless Sign-On

After reading more about it - --auth-server-allowlist="*"' sets opposite of what I want, it allows any auth server configured to be used. I couldn't find a switch to disable seamless sign-on for chrome/edge.

[gerrelatanacio]

Hello I had the same issue, I tried the same suggestion above like clearing the cookies but none of it worked. However, I stumbled on a solution as follows:

1. In your machine, open this directory: C:/Users/%username%/appdata/local/ms-playwright/chromiumXXX(version)/chrome-win/
2. Delete the MEIPreload folder. It is like this folder contains a preloaded data file which contains the authentication details.

Not sure if this will work for you but hopefully it will. :)

P.S. I assume that since I installed the chromium browser via the macro command from VSCode - Playwright Test, this flow created something like an authentication file in the process. Just a hunch. :)