[BUG] Playwright cookie not written on file for auth on gitlab-ci

[HatemTemimi]

Hello, I am trying to implement playwright Auth on gitlab ci and i have nearly tried every way possible, this is not a server related problem since the cookie is being sent from the server, what i have tried: 1) using the before script and going through regular UI auth, kicks back to the auth page (somehow the authentified state is not preserved) Link to video from playwright report using the before all method 2) I went to global auth config setup, first started with the global setup which worked at first and then did not work on gitlab ci (not creating cookies file, therefore not storing anything), so i checked the docs and found this new way of doing auth link 3) Auth setup file as a test, and then setting a setup project as dependency for other projects, so i implemented that, and now i am getting a 200 reponse from the server with the cookie, but it is not set on the cookies file i am setting( you will find the server reponse in the ci photos, and the cookie is not written in the file).

I am a bit confused to what i am doing wrong,i know that the problem lies within the function that writes the state to the file, so i thought about writing a function that writes the cookie from the response to a file replacing the storage state function, but i thought first maybe i am doing something wrong and someone can point me at the right direction, any help is appreciated, thanks.

System info

• Playwright Version: [1.32.2]
• Operating System: [All]
• Browser: [All, Chromium, Firefox, WebKit]
• Other info:

Source code

• [x] I provided exact source code that allows reproducing the issue locally.

Config file

// playwright.config.ts
/* eslint-disable import/no-extraneous-dependencies */
import { defineConfig, devices } from '@playwright/test';
import { config } from 'dotenv';
import { resolve } from 'path';
// import { resolve } from 'path';

import vite from '../../vite.config';

config({ path: `${process.env.INIT_CWD}/../../.env.local` });

export default defineConfig({
    // outputDir: '/builds/estale/playwright',
    testDir: './views',
    /* Maximum time one test can run for. */
    timeout: 30 * 1000,
    expect: {
    /**
    * Maximum time expect() should wait for the condition to be met.
    * For example in `await expect(locator).toHaveText();`
    */
        timeout: 5000,
    },
    /* Run tests in files in parallel */
    fullyParallel: false,
    /* Fail the build on CI if you accidentally left test.only in the source code. */
    forbidOnly: !!process.env.CI,
    /* Retry on CI only */
    retries: process.env.CI ? 2 : 0,
    /* Opt out of parallel tests on CI. */
    workers: process.env.CI ? 1 : undefined,
    /* Reporter to use. See https://playwright.dev/docs/test-reporters */
    reporter: [['html', { open: 'never' }]],
    /* Global setup and teardown scripts. See https://playwright.dev/docs/test-advanced#global-setup-and-teardown */
    // globalSetup: require.resolve('./playwright.setup.ts'),
    // globalTeardown: require.resolve('./playwright.teardown'),
    /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
    use: {
        headless: false,
        actionTimeout: 0,
        trace: 'on-first-retry',
        video: 'on',
        // baseURL: 'http://127.0.0.1:3000',
        screenshot: 'on',
        // baseURL: import.meta.env.BASE_URL,
        // baseURL:`http://${vite['server'].host}:${vite['server'].port}`,

        // storageState: resolve(__dirname, 'playwright.cookies.json'),
        launchOptions: {
            slowMo: 1000,
        },
        ignoreHTTPSErrors: true,
    },

    /* Configure projects for major browsers */
    projects: [
        { name: 'setup', testMatch: /.*\.setup\.ts/ },
        {
            name: 'chromium',
            use: {
                ...devices['Desktop Chrome'],

                storageState: resolve('src/intranet/views', 'playwright.cookies.json'),
            },
            dependencies: ['setup'],
        },
        {
            name: 'firefox',
            use: {
                ...devices['Desktop Firefox'],

                storageState: resolve('src/intranet/views', 'playwright.cookies.json'),
            },

            dependencies: ['setup'],

        },
        {
            name: 'webkit',
            use: {
                ...devices['Desktop Safari'],

                storageState: resolve('src/intranet/views', 'playwright.cookies.json'),
            },

            dependencies: ['setup'],
        },

    ],
    webServer: [{
        command: 'npm run dev',
        url: `http://${vite['server'].host}:${vite['server'].port}`,
        // url: 'http://127.0.0.1:3000',
        timeout: 120 * 1000,
        reuseExistingServer: !process.env.CI,
    }],
});

Auth setup test file

/* eslint-disable import/no-extraneous-dependencies */
// auth.setup.ts
import { test as setup, expect } from '@playwright/test';
import { resolve } from 'path';

const authFile = resolve(__dirname, 'playwright.cookies.json');

// await requestContext.storageState({ path: config.projects[0].use.storageState as string });

// const authFile = 'playwright.cookies.json';
const login = 'XXX;
const motdepasse = 'YYY';

//old method of authentication using UI (does not set the cookie)
/* setup('authenticate', async ({ page }) => {
    // Perform authentication steps. Replace these actions with your own.
    await page.goto('http://127.0.0.1:3000/auth/signin');
    const mail = page.getByPlaceholder('Saisissez votre adresse e-mail');
    const mdp = page.getByPlaceholder('Saisissez votre mot de passe');
    const loginBtn = page.getByText('Se connecter');
    await mail.fill(login, { force: true });
    await mdp.fill(motdepasse);
    await loginBtn.click();
    await page.waitForLoadState('domcontentloaded');
    await page.context().storageState({ path: authFile }).then(() => {
        console.log('state written to file!');
    }).catch((err) => console.log(err));
    // const l = await page.context().storageState();
    // console.log('authentication done!', l);
});
*/

//api auth(does not set the cookie)
setup('authenticate', async ({ request }) => {
    // Send authentication request. Replace with your own.
    console.log('inside api auth!');
    const response = await request.post('http://docker:8080/api/login', {
        data: {
            email: login,
            password: motdepasse,
        },
        headers: {
            'Content-Type': 'application/json',
        },
    });
    expect(response.ok()).toBeTruthy();
    expect(response.status()).toBe(200);
    console.log('response from auth test: ', await response.json());
    // await request.storageState({ path: authFile });
    await request.storageState({ path: authFile }).then(() => {
        console.log('state written to file!');
    }).catch((err) => console.log(err));
    console.log(response);
    console.log('Authentication done!');
});

Test file (self-contained)

const authFile = resolve(__dirname, 'playwright.cookies.json');
// test.use({ storageState: authFile });

test('Visit Intranet', async ({ page }) => {
    console.log(`cookies: ${await page.context().cookies()}`);
    await page.goto('http://127.0.0.1:3000/intranet/');
    await page.waitForLoadState('load');
});

Steps

• [Authentication]
• [Running the tests]

Expected

[The cookie should be written in the playwright.cookies.json file, and the tests authentified]

Actual

[The browser kicks back to the login page(using before script / global setup / auth.setup.ts), the cookie is also not set]

[AlexKorollev]

Same problem

[pavelfeldman]

I assume things work for you locally and only fail on gitlab-ci. If that is the case, you should look for the problem on the gitlab side.

[maissenayed]

Same problem

[HatemTemimi]

I am pretty confident the unexpected behavior is coming from this function await request.storageState({ path: authFile }).then(() => { console.log('state written to file!'); }).catch((err) => console.log(err)); If you check my ci log, you will see that the promise returned by request.storageState() was resolved and it went into the then() block, therefore doing the console statement. To my knowledge, this is obviously an unexpected behvaior, following the log, the function did create the file, but it did not write the cookie to it, so an error should be returned, and the promise should not be resolved. Aside from that, authenticating with UI from within the script kicks the browser back to the authentication page as seen in the video. Any insight on this is welcome.

[Silimar]

I'm facing the same issue.

I think it's really odd that we can still be facing this kind of issue in our day and age

[pavelfeldman]

the function did create the file, but it did not write the cookie to it

What is in the file though? And why do you think it should contain a cookie?

[pavelfeldman]

Most importantly, could you confirm it works locally and only fails on gitlab?

[HatemTemimi]

To answer your first question, the content of the file, as printed in the second photo i have attached, is actually a json object containing two empty arrays named "cookies" and "origins", the cookie array should contain a cookie object written by calling await request.storageState({ path: authFile }), as per the docs, but it ain't on the gitlab-ci. I think it should contain a cookie, because when it worked for me in local(which answers your second question, yes it worked locally on playwright 1.30.0), the playwright.cookies.json file was populated with a cookie object. Since the problem emerged, i created a function that writes the auth state to the file, and then called the auth file via test.use({ storageState: authFile }), the cookie was loaded in the context, which i have verified by doing console.log(await page.context().cookies()); from within the test. here is the result:

Report Video

[vancleef-c137]

Happening to me too and can't seem to find a fix.

[HatemTemimi]

UPDATE Okay so i have rerun the configuration on local machine with playwright 1.32.2 and a stripped playwright.config.ts file containing only necessary configuration. I have started by creating a sample test that authenticates the user via the UI and then redirects a statical UI from our website SAMPLE TEST

test('Visit Intranet', async ({ page }) => {
  await page.goto('http://127.0.0.1:3000/auth/signin');
  const mail = page.getByPlaceholder('Saisissez votre adresse e-mail');
  const mdp = page.getByPlaceholder('Saisissez votre mot de passe');
  const loginBtn = page.getByText('Se connecter');
  await mail.fill(login, { force: true });
  await mdp.fill(motdepasse);
  await loginBtn.click();
  await page.waitForLoadState('load');
  await page.goto('http://127.0.0.1:3000/');
  await page.waitForLoadState('load');
  console.log(await page.context().cookies()});
  await page.goto('http://127.0.0.1:3000/intranet/dashboard');
  await page.waitForLoadState('load');

});

And the result is the browser kicking back to the auth page as seen in the report video:

Local auth report

USING AUTH.SETUP.TS I have updated the playwright.config.ts to support the new auth dependency project with the auth.setup.ts, the file is created and the cookie file is populated with the cookie object, the browser context has the cookie, but the browser still kicks back to the auth page.

So maybe i am missing something regarding the global configuration of playwright 1.32.2 ? Any help is appreciated.

[pavelfeldman]

In the snippet above, there is nothing about the global configuration. I can see that when you log in and quickly navigate to the page again, you are still redirected to the logon page.

That's how your app behaves. You can open this page in a browser (incognito) and I assume the same thing will happen. I think what happens is that your app is persisting logged in state in a storage, lazily and you navigate off the page too quickly. All of those waitForLoadState calls are a noop, the page is already loaded.

To validate this assumption, you can wait for a couple of seconds after you log in. If that is the case, use page.waitForFunction or expect.poll to fetch localstorage or cookies from the page and wait for them to be there before you navigate off the lgon page.

[HatemTemimi]

In the snippet above, there is nothing about the global configuration. I can see that when you log in and quickly navigate to the page again, you are still redirected to the logon page.

That's how your app behaves. You can open this page in a browser (incognito) and I assume the same thing will happen. I think what happens is that your app is persisting logged in state in a storage, lazily and you navigate off the page too quickly. All of those waitForLoadState calls are a noop, the page is already loaded.

To validate this assumption, you can wait for a couple of seconds after you log in. If that is the case, use page.waitForFunction or expect.poll to fetch localstorage or cookies from the page and wait for them to be there before you navigate off the lgon page.

I have tested your assumption locally, and polling the UI auth process worked!

test.beforeEach(async ({ page }) => {
  await expect.poll(async () => {
    await page.goto('/auth/signin');
    const mail = page.getByPlaceholder('Saisissez votre adresse e-mail');
    const mdp = page.getByPlaceholder('Saisissez votre mot de passe');
    const loginBtn = page.getByText('Se connecter');
    await mail.fill(login, { force: true });
    await mdp.fill(motdepasse);
    await loginBtn.click();
    await page.waitForLoadState('load');
    const locator = await page.getByText('Tableau de bord');
    await expect(locator).toBeVisible();
  }, {
    intervals: [5_000, 8_000, 10_000],
    timeout: 60_000
  }).toBeTruthy();
});

test('Visit Intranet', async ({ page }) => {
  await page.goto('/');
  await page.waitForLoadState('load');
  console.log(`${await page.context().cookies()}`);
  await page.goto('/intranet/');
  await page.waitForLoadState('domcontentloaded');
});

however the problem still occurs when using global authentification via auth.setup.ts configuration with the cookie being set in the browser context (logged from within the test) but the browser kicks back to the auth page, do you think i should also poll the api request ? here is my configuration when running the last test using api authentication with auth.setup.ts locally along with the playwright video report.

playwright.config.ts

/* eslint-disable import/no-extraneous-dependencies */
import { defineConfig, devices } from '@playwright/test';
import { config } from 'dotenv';
import { resolve } from 'path';
// import { resolve } from 'path';

import vite from '../../vite.config';

config({ path: `${process.env.INIT_CWD}/../../.env.local` });

export default defineConfig({
  // outputDir: '/builds/estale/playwright',
  testDir: './views',
  /* Maximum time one test can run for. */
  timeout: 30 * 1000,
  expect: {
    /**
    * Maximum time expect() should wait for the condition to be met.
    * For example in `await expect(locator).toHaveText();`
    */
    timeout: 5000,
  },
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [['html', { open: 'never' }]],
  /* Global setup and teardown scripts. See https://playwright.dev/docs/test-advanced#global-setup-and-teardown */
   //globalSetup: require.resolve('./playwright.setup.ts'),
  // globalTeardown: require.resolve('./playwright.teardown'),
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    headless: false,
    //actionTimeout: 0,
    trace: 'on-first-retry',
    video: 'on',
    baseURL: 'http://127.0.0.1:3000',
    screenshot: 'on',
    // baseURL: import.meta.env.BASE_URL,
    // baseURL:`http://${vite['server'].host}:${vite['server'].port}`,

    // warning
    //! ERROR: storage state dir is not resolving correctly on ci
    // storageState: resolve('src/intranet/views', 'playwright.cookies.json'),
    //storageState: resolve(__dirname, 'playwright.cookies.json'),
    launchOptions: {
      slowMo: 1000,
    },
    //ignoreHTTPSErrors: true,
  },

  /* Configure projects for major browsers */
  projects: [
     { name: 'setup', testMatch: /.*\.setup\.ts/ },
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
        // ERROR: not reading storage state file on ci
        // storageState: resolve('src/intranet/views', 'playwright.cookies.json'),
      },
           dependencies: ['setup'],
    },
    {
      name: 'firefox',
      use: {
        ...devices['Desktop Firefox'],

        // storageState: resolve('src/intranet/views', 'playwright.cookies.json'),
      },

          dependencies: ['setup'],

    },
    {
      name: 'webkit',
      use: {
        ...devices['Desktop Safari'],

        // storageState: resolve('src/intranet/views', 'playwright.cookies.json'),
      },

         dependencies: ['setup'],
    },

  ],
  webServer: [{
    command: 'npm run dev',
    //url: `http://${vite['server'].host}:${vite['server'].port}`,
    url: 'http://127.0.0.1:3000',
    timeout: 120 * 1000,
    //reuseExistingServer: !process.env.CI,
  }],
});

auth.setup.ts

setup('authenticate', async ({ request }) => {
  // Send authentication request. Replace with your own.
  console.log('inside api auth!');
  const response = await request.post('http://127.0.0.1:8080/api/login', {
    data: {
      email: login,
      password: motdepasse,
    },
    headers: {
      'Content-Type': 'application/json',
    },
  });
  expect(response.ok()).toBeTruthy();
  await request.storageState({ path: authFile }).then(() => {
    console.log('state written to file!');
  }).catch((err) => console.log(err));

});

playwright.cookies.json populated by the auth script

{
  "cookies": [
    {
      "name": "estale-dev",
      "value": "PMMi1OlyWAM3cGYzIBWtnG5qI-VBQdKsO55QhZeCf5Q",
      "domain": ".127.0.0.1",
      "path": "/",
      "expires": 1681380834.531,
      "httpOnly": false,
      "secure": false,
      "sameSite": "Lax"
    }
  ],
  "origins": []
}

test.spec.ts

import { test, expect } from '@playwright/test';
import { config } from 'dotenv';
import { resolve } from 'path';

config();

const authFile = resolve('src/intranet/', 'playwright.cookies.json');

test.use({ storageState: authFile });

test('Visit Intranet', async ({ page }) => {
  await page.goto('/');
  await page.waitForLoadState('load');
  console.log(`${await page.context().cookies()}`);
  await page.goto('/intranet/collaborator/eb292bd7-95db-48a9-bbf9-1bb7376ff366/dashboard');
  await page.waitForLoadState('domcontentloaded');
  const locator = await page.getByText('Tableau de bord');
  await expect(locator).toBeVisible();

});

The browser context having the cookie object from within the tests

Report with authentication not working correctly

ba84bb69b5386756680acac73fc1c3f17709cde9.webm

With the playwright.cookies.json populated with a cookie from the server and passed down to the tests via test.use() why is the browser not authentified ? Am i missing something ?

[fedorovichpavel]

I had a similar problem. It's not a playwright problem, it's how the browser handles cookies. Just set up proxy rewrite from http://docker:8080/ to http://localhost:3000/test_api (or whatever domain you use for frontend). This will be on the same domain and will work as you expect.

I used Next.Js in my case it was something like this: next.config.js

const { i18n } = require('./next-i18next.config');
const { BREAK_POINTS } = require('./src/constants/breakpoints');

const nextConfig = {
    reactStrictMode: true,
    output: 'standalone',
    i18n,
    images: {
        deviceSizes: Object.values(BREAK_POINTS),
        domains: [process.env.NEXT_PUBLIC_API_HOST],
    },
    async rewrites() {
        const path_rewrites = []
        if (!!process.env.CI) {
            path_rewrites.push({
                source: '/test_api/:path*',
                destination: 'http://docker:8080/:path*',
            })
        }
        return path_rewrites
    },
};

module.exports = nextConfig;

There is a similar solution for Vite: https://vitejs.dev/config/server-options.html#server-proxy

[HatemTemimi]

So the problem is originating from http://docker:8080 and not from the playwright config ? I have added this to my vite.config.ts but it doesn't seem to hit the spot, auth still fails.

 server: {
        port,
        host,
        proxy: {
            '^/auth.*$': {
                target: `${url}/auth`,
                rewrite: () => '/',
            },
            '^/intranet.*$': {
                target: `${url}/intranet`,
                rewrite: () => '/',
            },
            '^/extranet.*$': {
                target: `${url}/extranet`,
                rewrite: () => '/',
            },

            '^/$': `${url}/auth`,
//added these lines
            'http://127.0.0.1:3000': {
                target: 'http://docker:8080',
                changeOrigin: true,
            },
        },
    },

[fedorovichpavel]

So the problem is originating from http://docker:8080 and not from the playwright config ?

Yes, because of the different domains for frontend and api. Try using a proxy, I think it will work for you as well.

[fedorovichpavel]

So the problem is originating from http://docker:8080 and not from the playwright config ? I have added this to my vite.config.ts but it doesn't seem to hit the spot, auth still fails.

 server: {
        port,
        host,
        proxy: {
            '^/auth.*$': {
                target: `${url}/auth`,
                rewrite: () => '/',
            },
            '^/intranet.*$': {
                target: `${url}/intranet`,
                rewrite: () => '/',
            },
            '^/extranet.*$': {
                target: `${url}/extranet`,
                rewrite: () => '/',
            },

            '^/$': `${url}/auth`,
//added these lines
            'http://127.0.0.1:3000': {
                target: 'http://docker:8080',
                changeOrigin: true,
            },
        },
    },

It should be something like this:

'/api': {
        target: 'http://docker:8080',
        changeOrigin: true,
        rewrite: (path) => path.replace(/^\/api/, ''),
      },

or

'/api': {
        target: 'http://docker:8080/api',
        changeOrigin: true,
        rewrite: (path) => path.replace(/^\/api/, ''),
      },

if you have api prefix

In your tests you should make api calls from http://frontend_domain/api

[HatemTemimi]

So the problem is originating from http://docker:8080 and not from the playwright config ? I have added this to my vite.config.ts but it doesn't seem to hit the spot, auth still fails.

 server: {
        port,
        host,
        proxy: {
            '^/auth.*$': {
                target: `${url}/auth`,
                rewrite: () => '/',
            },
            '^/intranet.*$': {
                target: `${url}/intranet`,
                rewrite: () => '/',
            },
            '^/extranet.*$': {
                target: `${url}/extranet`,
                rewrite: () => '/',
            },

            '^/$': `${url}/auth`,
//added these lines
            'http://127.0.0.1:3000': {
                target: 'http://docker:8080',
                changeOrigin: true,
            },
        },
    },

It should be something like this:

'/api': {
        target: 'http://docker:8080',
        changeOrigin: true,
        rewrite: (path) => path.replace(/^\/api/, ''),
      },

or

'/api': {
        target: 'http://docker:8080/api',
        changeOrigin: true,
        rewrite: (path) => path.replace(/^\/api/, ''),
      },

if you have api prefix

In your tests you should make api calls from http://frontend_domain/api

Sorry, it did not solve the problem, i also tried using an alias on the docker-dind

    - name: docker:20-dind
      alias: localhost

then using localhost for api calls, and it had connection refused on the api call. Also if this is any help, polling the beforeEach hook UI authentication worked for me on local, but not on gitlab-ci.

[fedorovichpavel]

Try to run your frontend part on CI and then check proxy set up. If you have health checker. By curl for example curl -v -X GET http://localhost:3000/api/health. Make sure that you set it up correctly.

[pavelfeldman]

Since this is getting offtopic and no longer corelates with the original issue title, I'll close it.

I have tested your assumption locally, and polling the UI auth process worked!

This is now what I meant. You should only login once. But then you should use polling to check that necessary cookies and localStorage are persisted, because your app can choose to do that lazily, after a timeout (which can be considered a bug in the app btw).

[HatemTemimi]

Try to run your frontend part on CI and then check proxy set up. If you have health checker. By curl for example curl -v -X GET http://localhost:3000/api/health. Make sure that you set it up correctly.

This has solved the problem, thank you kind sir @fedorovichpavel! it was a docker in docker problem not a playwright problem, so i set up the proxy for http://docker:8080 through http://127.0.0.1:3000/apiand the UI auth worked fine!

[tomatobrown]

@fedorovichpavel and @HatemTemimi Seems like this conversation evolved and I'm trying to understand if it's the same issue that I'm running into. I'm using playwright/vite/k8/docker/jenkins to run tests in CI. I'm trying to get code coverage to report in CI and I'm struggling to get the code coverage strictly when running in CI.

When I run vite in 'preview' mode all my tests run great and I do have access to the window.store and cookie in playwright in CI however code coverage is not reporting.

When I run vite not in 'preview' mode in CI I do get code coverage but the test have issues because playwright doesn't have access to things like cookies and the store.

Is this related to your issue?