[Feature] allow client certificate selection and settings from Javascript

[frzme]

Similarly to https://github.com/puppeteer/puppeteer/issues/540

Currently when navigating to a page that requires client certificates and client certificates are available a popup is shown in Firefox and Chrome which asks to select which certificate to use. It would be beneficial to provide an API to select the correct certificate to use (or use none).

[janostgren]

I have solution based on Axios https agent

sön 25 juni 2023 kl. 16:40 skrev SNIGDHADEB SAMANTA < @.***>:

Tried with all the alternatives. None of them worked. This is a very serious problem. Please help.

— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright/issues/1799#issuecomment-1606117043, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG7INMES6XE6ZOAYCGYUFZ3XNBEXXANCNFSM4MIV2WZQ . You are receiving this because you were mentioned.Message ID: @.***>

[pschroeder89]

For MacOS:

#!/bin/bash

# This script is used to configure the Chromium browser to automatically select the certificate for the staging environment.
defaults write org.chromium.Chromium AutoSelectCertificateForUrls -array
defaults write org.chromium.Chromium AutoSelectCertificateForUrls -array-add -string '{"pattern":"https://<url>:<port>", "filter": {}}'

For Chrome, change org.chromium.Chromium to com.google.Chrome

[gauravkhuraana]

Anything for dotnet which can help here. The feature was requested 2 years back. Are there any plans to bring this feature soon. Any workaround which can help by selecting any of the available certificates

[Az8th]

Hello !

I also have a need for certificate authentification. Ideally to select one given his id, or at least automatically load the first to be found, as it is possible to programatically remove one.

So, is there any ETA for this feature to be added ? Or at least could you provide us a progress report ? This issue is really upvoted, has been open for 3 years and had multiple duplicates 😕

This is the only downside I encountered using Playwright and I am sure that if you give a list of the required work to make it possible, some members of the community (including me) would be glad to help you with this 😉

Thanks <3

[h3tz]

I would also be happy to use certificates in the already mentioned easy "cypress" way. Thanks

[pschroeder89]

Figured this out, and I found that many of the solutions here are outdated or were incomplete for my needs. Regardless of if a simpler way to specify the selection of the certificate is implemented (which would be very welcome), I feel that at least a few people in this thread are not installing the P12 cert correctly, hence this full writeup.

Context: In scenarios where TLS certificates/keys are required to access a particular page in Playwright (or Selenium, any UI framework really) that uses Chrome, Chrome needs those certificate(s)/key added to the nssdb database on Linux. For context, in our setup, certificates are added through a security policy on my company's employee Macs. When running tests locally, we just need to suppress the "Select a Certificate" popup and auto-select it (refer to the Mac Local Testing section below). However, our tests also run in Docker in CI, where there isn't an auto-installation of certificates from the security policy.

Step 1: Obtain a Long-Living Certificate By default, our certificates expire frequently, which necessitated the creation of a longer-living certificate to avoid frequent script/variable updates.

Step 2: Scripting the Certificate Installation and Configuration I wrote a script to automate the process of installing the necessary tools and configuring the certs (see below for the script and the Linux CI Testing section for a description of what it does). I currently use it as a bash script before the tests run in CI, but you could easily bake it into your own Docker image instead.

Step 3: Store Certs / Key in CI Env Vars Don't hardcode your certs into your repo, store them as env vars. For my use case, we have the CA cert, the server cert, an intermediate cert, and the pem key.

Here's a generic version of the script:

#!/bin/bash

# This script is required for internal tests that require TLS to run. 
# On Mac, it configures Chromium to auto-select the certificate for the staging environment.
# On Linux, it installs `certutil` and installs each cert and the key into the system certificates, 
#   then configures Chromium to auto-select the certificate for the staging environment.

if [[ "$OSTYPE" == "darwin"* ]]; then
    # Configure Chromium browser to automatically select the certificate.
    defaults write org.chromium.Chromium AutoSelectCertificateForUrls -array
    defaults write org.chromium.Chromium AutoSelectCertificateForUrls -array-add -string '{"pattern":"*", "filter": {}}'
elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
    # Install certutil
    apt-get -qq update && apt-get -qq install libnss3-tools

    # Write QA_CA_CERT, QA_CERT, QA_INTERMEDIATE_CERT, and QA_KEY values to cert files
    echo -e "$QA_CA_CERT" > /usr/local/share/ca-certificates/root_ca.crt
    echo -e "$QA_CERT" > /usr/local/share/ca-certificates/server.crt
    echo -e "$QA_INTERMEDIATE_CERT" > /usr/local/share/ca-certificates/intermediate_ca.crt
    echo -e "$QA_KEY" > /usr/local/share/ca-certificates/server.pem

    # Create nssdb directory for certutil
    mkdir -p $HOME/.pki/nssdb

    # Create new database
    certutil -N --empty-password -d sql:$HOME/.pki/nssdb

    # Add Server certificate
    certutil -A \
        -n "QA" \
        -t "TC,," \
        -d sql:$HOME/.pki/nssdb \
        -i /usr/local/share/ca-certificates/server.crt

    # Add Intermediate CA certificate
    certutil -A \
        -n "Intermediate CA" \
        -t "TC,," \
        -d sql:$HOME/.pki/nssdb \
        -i /usr/local/share/ca-certificates/intermediate_ca.crt

    # Add Root CA certificate
    certutil -A \
        -n "Root CA" \
        -t "TC,," \
        -d sql:$HOME/.pki/nssdb \
        -i /usr/local/share/ca-certificates/root_ca.crt

    # Create p12 file
    cat /usr/local/share/ca-certificates/server.crt /usr/local/share/ca-certificates/intermediate_ca.crt /usr/local/share/ca-certificates/root_ca.crt > cert-chain.txt
    openssl pkcs12 -export \
        -in cert-chain.txt \
        -inkey /usr/local/share/ca-certificates/server.pem \
        -out /usr/local/share/ca-certificates/qa.p12 \
        -name "QA" \
        -passout pass:""

    # Import the PKCS#12 file
    pk12util -i /usr/local/share/ca-certificates/qa.p12 -d sql:$HOME/.pki/nssdb -W ""

    # Update system CA certificates
    update-ca-certificates --fresh

    # Create auto_select_certificate.json in the Chromium policies directory
    mkdir -p /etc/chromium/policies/managed/
    echo '{"AutoSelectCertificateForUrls": ["{\"pattern\":\"*\",\"filter\":{}}"]}' > /etc/chromium/policies/managed/auto_select_certificate.json
fi

Mac Local Testing: For local testing on Mac, the script configures the Chromium browser to automatically select the certificate. If you have multiple p12 key/certs, you'll need to update the filter section.

Linux CI Testing: The script installs certutil, writes the certificates and key from env vars to files, creates a new nssdb database, adds the certificates to the database, creates and imports a PKCS#12 file from the certs and key to the nssdb database, and creates a JSON configuration file for Chromium to auto-select the certificate, thus preventing the "Select a Certificate" popup.

Caveat! The above only works in --headed mode OR using --headless=new as a Chrome arg, since the old headless mode does not support Policies, which is required for the auto-selection of the certificate. We only need get presented for the Select a Certificate popup on a specific page of our internal site, so we override the launchOptions strictly for that test file, but you could put it into your config's launchOptions instead if needed globally:

test.use({
  // We need to use Chrome's new headless mode in CI to use policy certificates
  launchOptions: {
    args: process.env.CI ? ['--headless=new'] : [],
  },
});

[konradekk]

[…] In scenarios where TLS certificates/keys are required to access a particular page in Playwright (or Selenium, any UI framework really) that uses Chrome .

Nice, thanks for sharing! 🙇🏻‍♂️

(Itʼd be nice to have a similar workaround for Firefox now…! 😳)

[pschroeder89]

(Itʼd be nice to have a similar workaround for Firefox now…! 😳)

This doc will help you: https://wiki.mozilla.org/NSS_Shared_DB_Howto Since we aren't merging an existing db, I think you can set the env var, then edit your FF profile's pkcs11.txt file to point to our database path, and it will look to the nssdb we define in that above script.

[enrialonso]

[…] In scenarios where TLS certificates/keys are required to access a particular page in Playwright (or Selenium, any UI framework really) that uses Chrome .

Nice, thanks for sharing! 🙇🏻‍♂️

(Itʼd be nice to have a similar workaround for Firefox now…! 😳)

Hi, in this repository, I'm sharing this for and include Firefox. If you could take a look and provide feedback, I'd appreciate it. It may not be as sophisticated as the response from @pschroeder89, but it works for me.

[ilorwork]

@pschroeder89 What about local Windows? or Windows server?

[Az8th]

I got some news from @mxschmitt during today's Playwright Happy Hour : There is no update about the feature being added, but it is still on the scope !

Thanks for not letting us apart, and if we can help you in any way to get this feature, please tell ;)

[Fazali-Illahi]

I got some news from @mxschmitt during today's Playwright Happy Hour : There is no update about the feature being added, but it is still on the scope !

Thanks for not letting us apart, and if we can help you in any way to get this feature, please tell ;)

The community is eagerly waiting for this feature. It would really be nice to see this feature. I have discussed this blocker with @mxschmitt when I was working on an internal MSFT project. As you may be aware a lot of teams at even at Microsoft are moving to playwright, this will be a blocker for many teams. This feature should be considered a priority. I understand this is a niche feature and probaby unique to playwright. I can't remember any tool having this feature. I have always seen ugly workarounds for this. And I am sure others will agree...

[Kremliovskyi]

@Fazali-Illahi

Cypress has it https://docs.cypress.io/guides/references/client-certificates

[Az8th]

I got some news from @mxschmitt during today's Playwright Happy Hour : There is no update about the feature being added, but it is still on the scope ! Thanks for not letting us apart, and if we can help you in any way to get this feature, please tell ;)

The community is eagerly waiting for this feature. It would really be nice to see this feature. I have discussed this blocker with @mxschmitt when I was working on an internal MSFT project. As you may be aware a lot of teams at even at Microsoft are moving to playwright, this will be a blocker for many teams. This feature should be considered a priority. I understand this is a niche feature and probaby unique to playwright. I can't remember any tool having this feature. I have always seen ugly workarounds for this. And I am sure others will agree...

Ugly may be too harsh, but they are definitely not usable in CI, as they only concern headed mode, and not every browser.

Plus it would permit easier multiple authentication. Storage states are great, but setup multiple accounts for each worker is not very flexible when you have the need to reuse the same tests with different accounts types/permissions, while just swapping certs could be done on the fly and even remove the need for setup steps.

[Fazali-Illahi]

@Az8th. I agree it's straight forward to make this for a single browser in headed mode, it becomes tedious in cross browser tests on CI. And someone also mentioned this feature is already in cypress. I will definitely give that a try.

[ilorwork]

I'm really sorry to inform you that after a whole month of searching for a workaround, I'm moving to Cypress... We have about 15 Automation projects that I wanted to migrate from Selenium/Cypress to Playwright. My own project which is the one that should lead the whole migration failed to workaround this auth-related obstacle. I'm very disappointed because I really wanted to lead this thing, and this is a rare opportunity in such a big company, not to mention that the whole company stack is based on Microsoft.

[h3tz]

Cypress is not the better option. Please have a look into nightwatch much more software developer like.

Ilor @.***> schrieb am So., 19. Nov. 2023, 15:31:

I'm really sorry to inform you that after a whole month of searching for a workaround, I'm moving to Cypress... We have about 15 Automation projects that I wanted to migrate from Selenium/Cypress to Playwright. My own project which is the one that should lead the whole migration failed to workaround this auth-related obstacle. I'm very disappointed because I really wanted to lead this thing, and this is a rare opportunity in such a big company, not to mention that the whole company stack is based on Microsoft.

— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright/issues/1799#issuecomment-1817872311, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQ7XGS233K4JRWSMR6FRX3YFIJ3BAVCNFSM4MIV2WZ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRG44DOMRTGEYQ . You are receiving this because you commented.Message ID: @.***>

[janostgren]

I have done a workaround based on the Node https agent and Axios.

sön 19 nov. 2023 kl. 15:44 skrev Andreas Hetz @.***>:

Cypress is not the better option. Please have a look into nightwatch much more software developer like.

Ilor @.***> schrieb am So., 19. Nov. 2023, 15:31:

I'm really sorry to inform you that after a whole month of searching for a workaround, I'm moving to Cypress... We have about 15 Automation projects that I wanted to migrate from Selenium/Cypress to Playwright. My own project which is the one that should lead the whole migration failed to workaround this auth-related obstacle. I'm very disappointed because I really wanted to lead this thing, and this is a rare opportunity in such a big company, not to mention that the whole company stack is based on Microsoft.

— Reply to this email directly, view it on GitHub < https://github.com/microsoft/playwright/issues/1799#issuecomment-1817872311>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABQ7XGS233K4JRWSMR6FRX3YFIJ3BAVCNFSM4MIV2WZ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRG44DOMRTGEYQ>

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright/issues/1799#issuecomment-1817875439, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG7INMCAVYSYUD6R24I3DQTYFILLJAVCNFSM4MIV2WZ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRG44DONJUGM4Q . You are receiving this because you were mentioned.Message ID: @.***>

[Xen0byte]

I have done a workaround based on the Node https agent and Axios.

Yeah, great, except that all the language-specific issues have been aggregated into this issue, so this workaround may only be valid for TypeScript users but not for anyone else. For instance, I raised this problem for C# and it's been closed and merged into this issue, and none of these workarounds apply to me, and on top of that all the workarounds here seem to be for TypeScript because the issue title has not been updated to reflect the aggregation.

[janostgren]

Ok

mån 20 nov. 2023 kl. 08:06 skrev Vlad Tănăsescu @.***>:

I have done a workaround based on the Node https agent and Axios.

Yeah, great, except that all the language-specific issues have been aggregated into this issue, so this workaround may only be valid for TypeScript users but not for anyone else. For instance, I raised this problem for C# and it's been closed and merged into this issue, and none of these workarounds apply to me, and on top of that all the workarounds here seem to be for TypeScript because the issue title has not been updated to reflect the aggregation.

— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright/issues/1799#issuecomment-1818349473, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG7INMCMT5TMIZKLGJPHAWDYFL6Q5AVCNFSM4MIV2WZ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRHAZTIOJUG4ZQ . You are receiving this because you were mentioned.Message ID: @.***>

[ilorwork]

@janostgren Plz, can you share it with me? I'm using TS too, and if I'll find a full solution that would be great! Share it here or anywhere you want, thanks for the help

[kelvinsleonardo]

The same happens to me; I have some automations using Java, but I don't have a solution for Playwright with certificate selection. Is there any workaround?

[pschroeder89]

My workaround for Chrome / FF above is at the OS level. That same method can be used on Windows via registry changes.

Sent from my iPhone

On Mon, Nov 20, 2023 at 12:06 AM Vlad Tănăsescu @.***> wrote:

I have done a workaround based on the Node https agent and Axios.

Yeah, great, except that all the language-specific issues have been aggregated into this issue, so this workaround may only be valid for TypeScript users but not for anyone else. For instance, I raised this problem for C# and it's been closed and merged into this issue, and none of these workarounds apply to me, and on top of that all the workarounds here seem to be for TypeScript because the issue title has not been updated to reflect the aggregation.

— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright/issues/1799#issuecomment-1818349473, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACSDKRB3XPSXQ2NWY37MFHLYFL6QXAVCNFSM4MIV2WZ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRHAZTIOJUG4ZQ . You are receiving this because you were mentioned.Message ID: @.***>

[janostgren]

It is for a customer but we can have chat and discuss the topic.

mån 20 nov. 2023 kl. 14:36 skrev Ilor @.***>:

@janostgren https://github.com/janostgren Plz, can you share it with me? I'm using TS too, and if I'll find a full solution that would be great! Share it here or anywhere you want, thanks for the help

— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright/issues/1799#issuecomment-1819077359, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG7INMGVEM65P4MZCQJIGV3YFNMEJAVCNFSM4MIV2WZ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRHEYDONZTGU4Q . You are receiving this because you were mentioned.Message ID: @.***>

[ilorwork]

@janostgren Ok, lets chat. What is your preferred chatting platform? I'm available on telegram username: @letsolve_it

[alexyablonskyi]

Hey community. Any news regarding this feature? It is really block me to use Playwright

[ilorwork]

Workaround (UI)

Hey guys, I've posted a question about this issue on StackOverFlow, and added a few workarounds I found:

1. For Windows & Chromium users that uses the AutoSelectCertificateForUrls Registry Chromium policy - link
2. Using keyboard key pressing via node-key-sender library (Java runtime required) - link

Quick note - I'm using TypeScript, but I'm sure there are similar ways/libraries for other languages

[ilorwork]

@alexyablonskyi Check my previous commend, hope you'll find something useful

[IgorZn]

I've had similar issue and as a workaround using axios and fixture of playwright

httpClient.js

import https from 'node:https';
import * as fs from 'node:fs';
import axios from 'axios;

export const httpInstance = axios.create({
        baseURL: "<some baseURL>",
        httpsAgent : new https.Agent({
                cert: fs.readFileSync('<your>.pem'),
                key: fs.readFileSync('<your>.key'),
                rejectUnauthorized: false
            })
})

subscription.js

expost class Subscription {
        constructor(httpClient) {
                this.req = httpClient
        }

        async createSubscription(){
                return this.req.post('/api/v4/<somthing>', <body>)
        }

test-extend.js

import { test as customTest } from "@playwright/test"
import { Subscription } from "./POM/subscription";
import { httpInstance} from "./POM/httpClient";

export const test = customTest.extend({
    subscriptionHelper: async ({ request }, use) => {
        const subscription = new Subscription(httpInstance)
        await use(subscription)
    },
})

api-test.spec.js

import { expect } from "@playwright/test";
import { test } from "../fixtures/test-extend"

test('My Test', async ({ subscriptionHelper}) => {
    await subscriptionHelper.createSubscription()
        .catch(e => console.log(e.response.data))
})

Done

[Az8th]

Thanks for this workaround @IgorZn Does it works for all browsers, including headless mode ?

[IgorZn]

Thanks for this workaround @IgorZn Does it works for all browsers, including headless mode ?

No, it doesn't work for UI

[standbyoneself]

I also need this feature. I don't want it in the cypress way, I wanna select certificate programmatically e.g. by name in each test 😤.

[h3tz]

since nothing happened. We proceed with Robotframework.

[callmekohei]

Hello(｡･ω･｡)ﾉ

I've managed to implement and find a solution using F# (.NET) instead of JavaScript. It works properly. For your reference:

  let private httpRequestAsync (clientCert: X509Certificate2) (request: IRequest) =
    Http.AsyncRequest(
        url = request.Url
      , httpMethod = request.Method
      , headers = (request.Headers |> Seq.map(fun x -> x.Key,x.Value))
      , customizeHttpRequest =
          fun req ->
            req.ClientCertificates.Add(clientCert) |> ignore
            req
    )

  let private routeResponseWithCertificateAsync (clientCert: X509Certificate2) (route:IRoute) =
    task {
      try
        let! httpResponse = httpRequestAsync clientCert (route.Request)
        let opt = new RouteFulfillOptions()
        opt.Headers <- httpResponse.Headers |> Map.toSeq |> Seq.map(fun (k,v) -> KeyValuePair(k,v))
        opt.BodyBytes <-
          match httpResponse.Body with
          | Text text    -> System.Text.Encoding.UTF8.GetBytes(text)
          | Binary bytes -> bytes
        opt.ContentType <- httpResponse.Headers.Item("Content-Type")
        opt.Status <- httpResponse.StatusCode |> Nullable
        do! route.FulfillAsync(opt)
      with _ ->
        do! route.AbortAsync()
    }
    :> Task

  [<EntryPointAttribute>]
  let main _ =
    task {

      let url = "https://foo..."
      let urlPattern = "**/bar/"
      let clientCert =
        let cn = "baz"
        Certfs.GetCertificateByCommonName cn Certfs.myStoreName.My

      let! browser = Playwright.CreateAsync()
      let! edge = browser.Chromium.LaunchAsync(BrowserTypeLaunchOptions(Channel="msedge",Headless=false))
      let! context = edge.NewContextAsync()
      do! context.RouteAsync(urlPattern,(routeResponseWithCertificateAsync clientCert))
      let! page = context.NewPageAsync()
      do! sample url page
    }
    |> Task.WaitAll
    0

[standbyoneself]

Hello(｡･ω･｡)ﾉ

I've managed to implement and find a solution using F# (.NET) instead of JavaScript. It works properly. For your reference:

  let private httpRequestAsync (clientCert: X509Certificate2) (request: IRequest) =
    Http.AsyncRequest(
        url = request.Url
      , httpMethod = request.Method
      , headers = (request.Headers |> Seq.map(fun x -> x.Key,x.Value))
      , customizeHttpRequest =
          fun req ->
            req.ClientCertificates.Add(clientCert) |> ignore
            req
    )

  let private routeResponseWithCertificateAsync (clientCert: X509Certificate2) (route:IRoute) =
    task {
      try
        let! httpResponse = httpRequestAsync clientCert (route.Request)
        let opt = new RouteFulfillOptions()
        opt.Headers <- httpResponse.Headers |> Map.toSeq |> Seq.map(fun (k,v) -> KeyValuePair(k,v))
        opt.BodyBytes <-
          match httpResponse.Body with
          | Text text    -> System.Text.Encoding.UTF8.GetBytes(text)
          | Binary bytes -> bytes
        opt.ContentType <- httpResponse.Headers.Item("Content-Type")
        opt.Status <- httpResponse.StatusCode |> Nullable
        do! route.FulfillAsync(opt)
      with _ ->
        do! route.AbortAsync()
    }
    :> Task

  [<EntryPointAttribute>]
  let main _ =
    task {

      let url = "https://foo..."
      let urlPattern = "**/bar/"
      let clientCert =
        let cn = "baz"
        Certfs.GetCertificateByCommonName cn Certfs.myStoreName.My

      let! browser = Playwright.CreateAsync()
      let! edge = browser.Chromium.LaunchAsync(BrowserTypeLaunchOptions(Channel="msedge",Headless=false))
      let! context = edge.NewContextAsync()
      do! context.RouteAsync(urlPattern,(routeResponseWithCertificateAsync clientCert))
      let! page = context.NewPageAsync()
      do! sample url page
    }
    |> Task.WaitAll
    0

Does it work when there are multiple certificates in pop-up?

I have tried something similar but the main problem that my route doesn't intercepted because my website has OIDC authentication so many redirects occure. Any workarounds?

[callmekohei]

@standbyoneself

Does it work when there are multiple certificates in pop-up? -> yes

I have tried something similar but the main problem that my route doesn't intercepted because my website has OIDC authentication so many redirects occure. Any workarounds? -> I'm not quite sure about OIDC, sorry. However, could it be possible by capturing cookies for each connection? If that's the case, it seems doable.

[standbyoneself]

@standbyoneself

Does it work when there are multiple certificates in pop-up? -> yes

I have tried something similar but the main problem that my route doesn't intercepted because my website has OIDC authentication so many redirects occure. Any workarounds? -> I'm not quite sure about OIDC, sorry. However, could it be possible by capturing cookies for each connection? If that's the case, it seems doable.

Thanks for your reply, I'll take a look.

I'll publish my Java solution if I'll make it work.

[ilorwork]

I'll add my solution again since it works perfectly and even better than Cypress built-in solution:

A great workaround for Windows & Chromium users - using Windows Registry AutoSelectCertificateForUrls Chromium Policy

I’m demonstrating on Chromium/Chrome, but it’s the same for every other Chromium-based browser. (you just need to figure the exact key path)

For the Playwright Chromium browser you need to create this registry key: HKEY_CURRENT_USER\\SOFTWARE\\Policies\\Chromium\\AutoSelectCertificateForUrls

Note: According to google-docs If you use the PC Chrome (i.e. channel: "chrome") you should use this path instead: HKEY_CURRENT_USER\\SOFTWARE\\Policies\\Google\\Chrome\\AutoSelectCertificateForUrls

with this value pattern:

"1"="{\"pattern\":\"<https://www.example.com\>",\"filter\":{\"ISSUER\":{\"CN\":\"certificate issuer name\", \"L\": \"certificate issuer location\", \"O\": \"certificate issuer org\", \"OU\": \"certificate issuer org unit\"}, \"SUBJECT\":{\"CN\":\"certificate subject name\", \"L\": \"certificate subject location\", \"O\": \"certificate subject org\", \"OU\": \"certificate subject org unit\"}}}"

In this example I'm executing the CMD reg add command using node-js exec function:

const key = "HKEY_CURRENT_USER\\SOFTWARE\\Policies\\Chromium\\AutoSelectCertificateForUrls";
const value = "1";
const data = `{\\"pattern\\":\\"${url}\\",\\"filter\\":{\\"SUBJECT\\":{\\"CN\\":\\"${certName}\\"}}}`;

exec(`reg add "${key}" /v "${value}" /d "${data}"`);

Now, navigate to your site and the certificate pop-up shouldn't pop-up.

[standbyoneself]

I'll add my solution again since it works perfectly and even better than Cypress built-in solution:

A great workaround for Windows & Chromium users - using Windows Registry AutoSelectCertificateForUrls Chromium Policy

I’m demonstrating on Chromium/Chrome, but it’s the same for every other Chromium-based browser. (you just need to figure the exact key path)

For the Playwright Chromium browser you need to create this registry key: HKEY_CURRENT_USER\\SOFTWARE\\Policies\\Chromium\\AutoSelectCertificateForUrls

Note: According to google-docs If you use the PC Chrome (i.e. channel: "chrome") you should use this path instead: HKEY_CURRENT_USER\\SOFTWARE\\Policies\\Google\\Chrome\\AutoSelectCertificateForUrls

with this value pattern:

"1"="{\"pattern\":\"<https://www.example.com\>",\"filter\":{\"ISSUER\":{\"CN\":\"certificate issuer name\", \"L\": \"certificate issuer location\", \"O\": \"certificate issuer org\", \"OU\": \"certificate issuer org unit\"}, \"SUBJECT\":{\"CN\":\"certificate subject name\", \"L\": \"certificate subject location\", \"O\": \"certificate subject org\", \"OU\": \"certificate subject org unit\"}}}"

In this example I'm executing the CMD reg add command using node-js exec function:

const key = "HKEY_CURRENT_USER\\SOFTWARE\\Policies\\Chromium\\AutoSelectCertificateForUrls";
const value = "1";
const data = `{\\"pattern\\":\\"${url}\\",\\"filter\\":{\\"SUBJECT\\":{\\"CN\\":\\"${certName}\\"}}}`;

exec(`reg add "${key}" /v "${value}" /d "${data}"`);

Now, navigate to your site and the certificate pop-up shouldn't pop-up.

Thanks. It works not for Windows only, but it is a not flexy solution.

I build a platform with RBAC and I have about 10-20 certificates for QA stand (multiple roles).

Suppose, if I use Java, I wanna create the annotation @Cert(acc-1, acc-2, acc-3). It would run this test with 3 certificates, may be in parallel.

So it's about selecting certificates programmatically, wonder that it is not properly implemented by any framework.

P.S. Also this policy requires browser to be run headed which is also more expensive and more difficult especially in CI.

[callmekohei]

I think the solution using the registry is also very valuable. In fact, I was doing that too.However, not everyone can manipulate the registry.The method using F# prevents the pop-up from appearing in the first place, so I believe it has its own value.

[callmekohei]

I'm Sorry...

I mentioned in my previous post that I managed to get certificate authentication working with F# without touching the registry settings. Turns out, it was actually working because of some pre-existing registry settings I hadn't noticed. Sorry for the mix-up! Here's the code I was talking about:

  let private httpRequestAsync (clientCert: X509Certificate2) (request: IRequest) =
    Http.AsyncRequest(
        url = request.Url
      , httpMethod = request.Method
      , headers = (request.Headers |> Seq.map(fun x -> x.Key,x.Value))
      , customizeHttpRequest =
          fun req ->
            req.ClientCertificates.Add(clientCert) |> ignore
            req
    )

  let private routeResponseWithCertificateAsync (clientCert: X509Certificate2) (route:IRoute) =
    task {
      try
        let! httpResponse = httpRequestAsync clientCert (route.Request)
        let opt = new RouteFulfillOptions()
        opt.Headers <- httpResponse.Headers |> Map.toSeq |> Seq.map(fun (k,v) -> KeyValuePair(k,v))
        opt.BodyBytes <-
          match httpResponse.Body with
          | Text text    -> System.Text.Encoding.UTF8.GetBytes(text)
          | Binary bytes -> bytes
        opt.ContentType <- httpResponse.Headers.Item("Content-Type")
        opt.Status <- httpResponse.StatusCode |> Nullable
        do! route.FulfillAsync(opt)
      with _ ->
        do! route.AbortAsync()
    }
    :> Task

  [<EntryPointAttribute>]
  let main _ =
    task {

      let url = "https://foo..."
      let urlPattern = "**/bar/"
      let clientCert =
        let cn = "baz"
        Certfs.GetCertificateByCommonName cn Certfs.myStoreName.My

      let! browser = Playwright.CreateAsync()
      let! edge = browser.Chromium.LaunchAsync(BrowserTypeLaunchOptions(Channel="msedge",Headless=false))
      let! context = edge.NewContextAsync()
      do! context.RouteAsync(urlPattern,(routeResponseWithCertificateAsync clientCert))
      let! page = context.NewPageAsync()
      do! sample url page
    }
    |> Task.WaitAll
    0

Really sorry for sharing incorrect information earlier. This has been a good reminder of how important it is to fully understand all the elements behind a solution. Looking forward to learning more and contributing to the community. Thanks for your understanding!

@standbyoneself @ilorwork

[ilorwork]

Thanks for your hard work and honesty. any kind of solution being found its a great progress. I have to say that this registry solution I've posted saved my life... After months of searching and kinda wasting my and team's time, I've almost quit Playwright and already started migrating to Cypress, I was very disappointed! and in my few last desperate attempts - I found this solution.

[Az8th]

Unfortunately, all those workarounds are not compatible with the new MPT service. It seems almost everything that is related to certificates is folded in this issue, but I have the feeling that it doesn't help with comprehension nor solving.

Meanwhile, this issue is mentioned at almost every community event, and as explained by Debbie during one of the last Playwright Happy Hour, there are several issues that should be adressed independently, so maybe it could help to split them here too.

Here are, if I remember correctly, the different ones (feel free to correct me if there is any mistake/miscomprehension) :

• Add possibility to interact with the certificate choosing dialog
• Add possibility to provide the certificate given its file to the context object (and ability to skip the choosing dialog if possible)
• Add a certificate manager that could permit to switch between multiple ones

[standbyoneself]

Hi there. Here is my solution. Based on fetching session cookies, so there is no popup shown, works in headless=true.

Steps: 1) Create SSLContext using sslcontext-kickstart-for-pem library 2) Make GET request to the website in order to fetch cookies 3) Add cookies to Playwright Context

PageTestFixtures.java

import java.lang.reflect.Method;
import java.nio.file.Paths;
import java.util.List;
import java.util.Optional;
import java.util.UUID;

import org.junit.jupiter.api.*;

import com.microsoft.playwright.*;
import com.microsoft.playwright.options.Cookie;

import io.qameta.allure.Allure;
import ru.sbrf.rmc.AuthClient;
import ru.sbrf.rmc.AuthClientImpl;
import ru.sbrf.rmc.BrowserFactory;
import ru.sbrf.rmc.BrowserFactoryImpl;
import ru.sbrf.rmc.annotations.Certificate;
import ru.sbrf.rmc.mappers.CookieMapperImpl;

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
public class PageTestFixtures {
    private Playwright playwright;
    private BrowserType browserType;
    private Browser browser;
    private BrowserContext context;
    protected Page page;
    private AuthClient authClient = new AuthClientImpl(new CookieMapperImpl());

    private String getCertificateName(TestInfo testInfo) throws NoSuchMethodException {
        Optional<Method> optionalMethod = testInfo.getTestMethod();

        if (optionalMethod.isEmpty()) {
            throw new NoSuchMethodException();
        }

        Method method = optionalMethod.get();
        Certificate annotation = method.getAnnotation(Certificate.class);
        return annotation.value();
    }

    private void updateAllureTestNameAndHistoryId(String baseName) {
        Allure.getLifecycle().updateTestCase(testResult -> {
            String name = String.format("%s: %s", baseName, testResult.getName());
            String historyId = UUID.nameUUIDFromBytes(name.getBytes()).toString();
            testResult.setName(name);
            testResult.setHistoryId(historyId);
        });
    }

    @BeforeAll
    public void launchBrowser() {
        playwright = Playwright.create();
        BrowserFactory browserFactory = new BrowserFactoryImpl(playwright);
        String browserName = System.getenv().getOrDefault("BROWSER", "chromium");
        browserType = browserFactory.createBrowserType(browserName);
        BrowserType.LaunchOptions launchOptions = new BrowserType.LaunchOptions();
        Optional<String> browserPath = Optional.ofNullable(System.getenv("BROWSER_PATH"));
        if (browserPath.isPresent()) {
            launchOptions.setExecutablePath(Paths.get(browserPath.get()));
        }
        browser = browserType.launch(launchOptions);
    }

    @AfterAll
    public void closeBrowser() {
        playwright.close();
    }

    @BeforeEach
    public void createContextAndPage(TestInfo testInfo) {
        updateAllureTestNameAndHistoryId(String.format("%s %s", browserType.name(), browser.version()));
        context = browser.newContext(new Browser.NewContextOptions()
            .setBaseURL(System.getenv().getOrDefault("BASE_URL", "https://rmp-ift.sberbank.ru/lenta/"))
            .setIgnoreHTTPSErrors(true));
        page = context.newPage();

        try {
            String certificateName = getCertificateName(testInfo);
            List<Cookie> cookies = authClient.getSessionCookies(certificateName);
            context.addCookies(cookies);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @AfterEach
    public void closeContext() {
        context.close();
    }
}

AuthClientImpl.java

package ru.sbrf.rmc;

import java.io.IOException;
import java.net.CookieManager;
import java.net.CookiePolicy;
import java.net.CookieStore;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpRequest.BodyPublishers;
import java.net.http.HttpResponse.BodyHandlers;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;

import javax.net.ssl.X509ExtendedKeyManager;
import javax.net.ssl.X509ExtendedTrustManager;

import com.microsoft.playwright.options.Cookie;

import nl.altindag.ssl.SSLFactory;
import nl.altindag.ssl.util.PemUtils;
import ru.sbrf.rmc.mappers.CookieMapper;

public class AuthClientImpl implements AuthClient {
    private CookieMapper cookieMapper;

    public AuthClientImpl(CookieMapper cookieMapper) {
        this.cookieMapper = cookieMapper;
    }

    private SSLFactory getSSLFactory(String certificateName) {
        Path currentDir = Paths.get("ssl");
        Path fullPath = currentDir.toAbsolutePath();
        X509ExtendedKeyManager keyManager = PemUtils.loadIdentityMaterial(fullPath.resolve(certificateName + ".crt"), fullPath.resolve(certificateName + ".key"));
        X509ExtendedTrustManager trustManager = PemUtils.loadTrustMaterial(fullPath.resolve("ca.pem"));
        return SSLFactory.builder().withIdentityMaterial(keyManager).withTrustMaterial(trustManager).build();
    }

    public void sendRequest(String url, String certificateName, CookieManager cookieManager) throws IOException, InterruptedException, URISyntaxException {
        SSLFactory sslFactory = getSSLFactory(certificateName);
        HttpRequest request = HttpRequest.newBuilder().uri(new URI(url)).method("GET", BodyPublishers.noBody()).setHeader("Accept", "*/*").build();
        HttpClient client = HttpClient.newBuilder().cookieHandler(cookieManager).followRedirects(HttpClient.Redirect.NORMAL).sslParameters(sslFactory.getSslParameters()).sslContext(sslFactory.getSslContext()).build();
        client.send(request, BodyHandlers.ofString());
    }

    public List<Cookie> getSessionCookies(String certificateName) throws IOException, InterruptedException, URISyntaxException {
        CookieManager cookieManager = new CookieManager(null, CookiePolicy.ACCEPT_ORIGINAL_SERVER);
        CookieStore cookieStore = cookieManager.getCookieStore();
        sendRequest("https://rmp-ift.sberbank.ru", certificateName, cookieManager);
        return cookieMapper.httpCookiesToPlaywrightCookies(cookieStore.getCookies());
    }
}

Certificates are stored under ssl folder (don't forget to add it to .gitignore).

CookieMapperImpl.java

package ru.sbrf.rmc.mappers;

import java.net.HttpCookie;
import java.util.List;

import com.microsoft.playwright.options.Cookie;

public class CookieMapperImpl implements CookieMapper {
    public Cookie httpCookieToPlaywrightCookie(HttpCookie httpCookie) {
        return new Cookie(httpCookie.getName(), httpCookie.getValue())
            .setDomain(httpCookie.getDomain())
            .setPath(httpCookie.getPath())
            .setHttpOnly(httpCookie.isHttpOnly())
            .setSecure(httpCookie.getSecure());
    }

    public List<Cookie> httpCookiesToPlaywrightCookies(List<HttpCookie> httpCookies) {
        return httpCookies.stream().map(this::httpCookieToPlaywrightCookie).toList();
    }
}

Certificate.java

package ru.sbrf.rmc.annotations;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Certificate {
    String value();
}

Using in tests:

AdminPageTests.java

import java.util.Date;

import org.junit.jupiter.api.*;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;

import io.qameta.allure.Issue;
import ru.sbrf.rmc.annotations.Certificate;
import ru.sbrf.rmc.annotations.TestCase;
import ru.sbrf.rmc.annotations.TestCases;
import ru.sbrf.rmc.models.AdminPagePath;
import ru.sbrf.rmc.pages.AdminPage;

@DisplayName("Админка")
public class AdminPageTests extends PageTestFixtures {
    private AdminPage adminPage;

    @BeforeEach
    public void createPage() {
        adminPage = new AdminPage(page);
    }

    @Nested
    @DisplayName("Объекты")
    class Subjects {
        private void shouldCreateAndDeleteSubject(String prefix, String searchQuery) {
            long timestamp = new Date().getTime();
            String name = prefix + "-" + timestamp;
            adminPage.goTo(AdminPagePath.SUBJECTS);
            adminPage.goToSubjectForm();
            adminPage.fillSubjectName(name);
            adminPage.fillSubjectSearchQuery(searchQuery);
            adminPage.submitSubjectForm();
            adminPage.waitFor(AdminPagePath.SUBJECTS);
            adminPage.checkIfSubjectIsVisible(name);
            adminPage.deleteSubject(name);
        }

        @ParameterizedTest(name = "{displayName}")
        @Certificate("curuser-6")
        @Issue("RMC-6335")
        @TestCases({@TestCase("RMC-T6354"), @TestCase("RMC-T6346")})
        @CsvSource({"Subject,playwright OR selenium"})
        @DisplayName("Создание и удаление объекта с ролью \"Бизнес-администратор\"")
        public void shouldCreateAndDeleteSubjectByFeedAdmin(String prefix, String searchQuery) {
            shouldCreateAndDeleteSubject(prefix, searchQuery);
        }

        @ParameterizedTest(name = "{displayName}")
        @Certificate("curuser-10")
        @Issue("RMC-6335")
        @TestCases({@TestCase("RMC-T6355"), @TestCase("RMC-T6347")})
        @CsvSource({"Subject,playwright OR selenium"})
        @DisplayName("Создание и удаление объекта с ролью \"Куратор процесса\"")
        public void shouldCreateAndDeleteSubjectByProcessManager(String prefix, String searchQuery) {
            shouldCreateAndDeleteSubject(prefix, searchQuery);
        }
    }
}

[Az8th]

@mxschmitt Deep thanks for adding this to the next release ! Sorry for every bit of impatience and frustation you have faced from users concerning this feature request.

Is there any ETA for 1.46 ? Also by providing multiple certificates to the context, how can we handle the certificate choice dialog and will providing only one will skip it ?