[BUG] Can't run Playwright in Nix

[VaasRamsay]

Context:

• Playwright Version: 1.8.0
• Operating System: Linux
• Node.js version: 12
• Browser: All

Describe the bug

Running Playwright in Nix Shell throws missing dependencies error. Even after explicitly adding the missing dependencies suggested by "launch doctor", the same error is thrown.

Error: browserType.launch: Host system is missing dependencies!
--
  | Missing libraries we didn't find packages for:
  | libexpat.so.1
  | libxshmfence.so.1

Probably, I'm hitting this error from here. Nix has it's own ldconfig and it's not found in sbin.

[aslushnikov]

@SamBreaksThings we don't support Nix at the moment and this is the first request we've received regarding it so far.

So until there's a huge demand for Nix support, this will be low-pri for us. Is there something easy we can do to enable you use Playwright on Nix?

[uri-canva]

Is it possible to make the linked workaround configurable: https://github.com/microsoft/playwright/blob/a9faa9c941fa0b0383d68dff2fb7ef825bd258ea/src/server/validateDependencies.ts#L247-L250?

Note the commit message of https://github.com/aslushnikov/playwright/commit/aa3019bc8225abd463602481e3362fc61ea33b40

To the best of my knowledge, ldconfig is always located at /sbin.

This is one case where it's not located there, and loading it from there leads to broken behaviour.

[joscha]

Is it possible to make the linked workaround configurable:

maybe via an env var await spawnAsync(process.env.LDCONFIG_PATH || '/sbin/ldconfig', ['-p'], {});

? Or if that is an unwanted API it would also be thinkable to make it a fallback from the generic ldconfig (w/o qualified path, e.g. command -v ldconfig or which ldconfig) and if that one is not found to fall back to /sbin/ldconfig. Or if we want to make the speed sacrifice for more exotic users of this package if could be in the catch block of the /sbin/ldconfig spawn and introspect the error and retry with ldconfig from path?

[uri-canva]

Alternatively an option to entirely disable validating dependencies would work even better: I'm noticing ldconfig is only used to provide an error message suggesting how to install missing dependencies.

[aslushnikov]

@uri-canva @joscha Is this the only thing that breaks Playwright on Nix? In this case, would symlinking the right ldconfig to /sbin/ldconfig solve it on your end?

[joscha]

A system can have multiple nix profiles. Which one is used is dependent on the environment started. These virtualizations only affect the PATH (and a few other things like env) but not the file system, so changing a symlink is not really idiomatic as you would suddenly bind the static filesystem to a specific, arbitrary nix profile. On CI (where we intend to run this) it would mean that the symlink would point to a random profile or possibly nothing at all. I understand that an env var driving this might be an API we don't want to introduce but what do you think about falling back from the entry in PATH to the hardcoded location or skipping use of ldconfig altogether like @uri-canva suggested?

[aslushnikov]

@joscha if this is the only thing that doesn't work on Nix - then sure, we can come up with something as long as it's non-intrusive for the codebase & API.

But is this the only thing that prevents you from running on NIX? For example, I'll be surprised if our browser builds launch on NIX. Have you tried running them?

[uri-canva]

The browsers require a wrapper around them to launch on nix: https://github.com/ludios/nixos-playwright

[aslushnikov]

The browsers require a wrapper around them to launch on nix: https://github.com/ludios/nixos-playwright

@uri-canva impressive!

So to double-check: the ldconfig is the last thing that stops you folks from using playwright on nix?

Would you be open to send a PR that detects *NIX system and disables launch doctor there? I cannot guarantee we land it, but we'll discuss it. I'd appreciate if it's as non-invasive as possible.

P.S. I apologize for all the questions - I have zero experience with NIX systems, and playing with it in docker was painful - there's even no bash 🤷‍♂️

[uri-canva]

We can make it agnostic to nix: look if ldconfig is on PATH, if not try /sbin/ldconfig, if that doesn't exist either fail?

[aslushnikov]

This sounds good to me!

[olebedev]

I have made a PR that allows disabling host requirements validation, please have a look https://github.com/microsoft/playwright/pull/5806.

[mxschmitt]

Since the PR for skipping the host validation got merged and there won't be official supported added we will close it for now, feel free to reopen / upvote it when there is a higher demand.

[kylegentle]

@olebedev , would you mind sharing a repo link (or gist) for the Nix derivation you use to wrap playwright's browser binaries as referenced in #5806? Or are you using the one linked above?

Apologies for commenting on an old issue; I considered sending an email, but figured asking in a public forum might help others with the same question.

[olebedev]

@olebedev , would you mind sharing a repo link (or gist) for the Nix derivation you use to wrap playwright's browser binaries as referenced in #5806? Or are you using the one linked above?

@kylegentle, no, I use my own-written Nix derivation. It's not just a code snippet so I need to go through the OSS request process within the company I work for, to make it public. I will keep you posted here.

[aabccd021]

Playwright package is now available in unstable channel. I got it working with nix flake by using following flake.nix and then running nix develop

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
  };

  outputs = { self, nixpkgs }: with nixpkgs.legacyPackages.x86_64-linux; {
    devShell.x86_64-linux = mkShell {
      buildInputs = [
        playwright
        playwright.browsers
      ];
      shellHook = ''
        export PLAYWRIGHT_BROWSERS_PATH=${playwright.browsers}
        export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=true
      '';
    };
  };
}

`flake.lock`

```json { "nodes": { "nixpkgs": { "locked": { "lastModified": 1666926733, "narHash": "sha256-+gYfOEnQVISPDRNoWm2VJD5OEuTUySt48RchLpvm61o=", "owner": "nixos", "repo": "nixpkgs", "rev": "f44ba1be526c8da9e79a5759feca2365204003f6", "type": "github" }, "original": { "owner": "nixos", "ref": "nixpkgs-unstable", "repo": "nixpkgs", "type": "github" } }, "root": { "inputs": { "nixpkgs": "nixpkgs" } } }, "root": "root", "version": 7 } ```

[FossPrime]

playwright.browsers

I have 22.11 with playwright installed through config... that may work for python, but packages that depend on playwright like Node Slidev still don't work and you'll get the dreaded message:

Host system is missing dependencies to run browsers. ║
║ Missing libraries:                                   ║
║     libgobject-2.0.so.0                              ║
║     libglib-2.0.so.0                                 ║
║     libnss3.so                                       ║
║     libnssutil3.so                                   ║
║     libsmime3.so                                     ║
║     libnspr4.so                                      ║
║     libatk-1.0.so.0                                  ║
║     libatk-bridge-2.0.so.0                           ║
║     libcups.so.2                                     ║
║     libgio-2.0.so.0                                  ║
║     libdrm.so.2                                      ║
║     libdbus-1.so.3                                   ║
║     libatspi.so.0                                    ║
║     libX11.so.6                                      ║
║     libXcomposite.so.1                               ║
║     libXdamage.so.1                                  ║
║     libXext.so.6                                     ║
║     libXfixes.so.3                                   ║
║     libXrandr.so.2                                   ║
║     libgbm.so.1                                      ║
║     libexpat.so.1                                    ║
║     libxcb.so.1                                      ║
║     libxkbcommon.so.0                                ║
║     libpango-1.0.so.0                                ║
║     libcairo.so.2                                    ║
║     libasound.so.2

[adam248]

I get the same error message... installed playwright via NixOS /etc/nixos/configuration.nix in users.users.<me>.packages = with pkgs; [playwright]; playwright install does download the 3 different browser drivers, with a warning BEWARE: your OS is not officially supported by Playwright; downloading fallback build. And finally, playwright open or any command like that causes this error (see above^). I am going to link this issue on the nixpkgs repo to see if we can get some help from the NixOS community...

[adam248]

Calling the heroes in: @techknowlogick @yrd @SuperSandro2000

as per https://github.com/NixOS/nixpkgs/issues/215450

[yrd]

playwright install does download the 3 different browser drivers, with a warning BEWARE: your OS is not officially supported by Playwright; downloading fallback build.

The Playwright package splits the browser bundle depending on the platform. For Linux, playwright.browsers is a derivation that emulates what you get when running playwright install, but with the Nixpkgs version of each browser. As @aabccd021 mentioned, you will need to set PLAYWRIGHT_BROWSERS_PATH to this derivation, instead of having Playwright download the precompiled browsers. Try something like this:

export PLAYWRIGHT_BROWSERS_PATH=$(nix build --print-out-paths nixpkgs#playwright.browsers)
nix shell nixpkgs#playwright
playwright open nixos.org

(Add --no-link to the nix build step if you don't wan't the result symlink to the browsers derivation). I just tested this using NixOS/nixpkgs@5a350a8f31bb7ef0c6e79aea3795a890cf7743d4 and it seems to work:

If you're on Darwin, playwright.browsers will be the same bundle you get when running playwright install (the pre-built binaries by the Playwright team, including their patches), just as an FOD so we get the reproducibility. On Mac, using Playwright's install script as @adam248 described should work as well, but playwright.browsers in combination with the environment variable will keep everything in the Nix store.

[yrd]

One more note when using our browser bundle on Linux instead of the Playwright-provided one: this does mean that you won't be getting any Playwright-specific browser patches. Maybe someone from the Playwright team could weigh in on the implications here?

[SuperSandro2000]

Applying a 3000 or 23000 line patch is guaranteed to break at some point. Unless those patches get split up into smaller ones, we cannot maintainable apply them to a source build unless we fetch the same exact version.

[yrd]

I just don't know if there are any Playwright Features we're currently missing out on by using the regular browser builds… But I do agree with @SuperSandro2000 that it's infeasible for us to include all patches. Maybe there are some parts which could be interesting to cherry-pick, but I'm not versed enough in Playwright's internals to say anything about that.

[pbek]

Currently, this shell.nix is working for me:

{ pkgs ? import <nixpkgs> {} }:
  pkgs.mkShell {
    # nativeBuildInputs is usually what you want -- tools you need to run
    nativeBuildInputs = with pkgs; [
      playwright-driver.browsers
    ];

    shellHook = ''
      export PLAYWRIGHT_BROWSERS_PATH=${pkgs.playwright-driver.browsers}
      export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=true
    '';
}

[ManUtopiK]

@pbek It works also for me with node package @playwright/test version 1.40.1 with the same shell.nix. I updated to 1.41.1 and now I can't launch playwright : Error No chromium-based browser found on the system. I've downgraded for now...

[pbek]

I already added the version matching in the nix wiki last week, because I had a similar issue. 😅

[ml21]

Sorry, if my questions are stupid - I'm newbie in nix.

I already added the version matching in the nix wiki last week, because I had a similar issue. 😅

I tried to find it in wiki https://nixos.wiki/wiki/Playwright , but no version matching there. Where can I find this version matching?

And one more question please - in wiki:

Note: Keep in mind that you need to use the same version of playwright in your node playwright project as in your nixpkgs, or else playwright will try to use browsers versions that aren't installed!

How can I specify playwright\playwright-driver.browsers version in my flake.nix?

For example, if my project uses playwright version 1.41 how can I specify right playwright\playwright-driver.browsers nixpkgs versions?

Here is my test flake with dev env

{
  description = "dev env";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
  };

  outputs = { self, nixpkgs }: let
    system = "x86_64-linux";
    pkgs = import nixpkgs { inherit system; };
  in {
      devShells.${system}.default = pkgs.mkShell {
      packages = with pkgs; [
        nodejs
        dotnet-sdk_8
        playwright
      ];

      PLAYWRIGHT_NODEJS_PATH = "${pkgs.nodejs}/bin/node";
      PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true;
      PLAYWRIGHT_BROWSERS_PATH="${pkgs.playwright-driver.browsers}";

      shellHook = ''
        echo "dev env started"
        fish && echo "exit dev env" && exit
      '';
    };
  };
}

[pbek]

Note: Keep in mind that you need to use the same version of playwright in your node playwright project as in your nixpkgs, or else playwright will try to use browsers versions that aren't installed!

I was talking about that note. 😉

I was looking at https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=playwright-driver, noting the version. Then updating my flake. Then installing exact that version with npm too.

[ml21]

Then installing exact that version with npm too.

What if I can't change the version in the project (dotnet in my case, not node)?

In this case, I need to find and specify a suitable version of playwright from nixpkgs.

How could I do that?

—- Updated: I think I found a way to find old versions: https://www.nixhub.io/packages/playwright

Anyway - @pbek thank you, your posts here and on discourse very useful for me

[pbek]

https://www.nixhub.io/packages/playwright

Oh, that's nice. I didn't know that service! And I didn't know "devbox". Did you try it, @ml21? Does it create a flake for your project to pin the version? Because that's what I would have done. Pin the nix version of playwright (by searching the github nixpkgs repository for the commit hash of exactly the version that I need) with a nix flake.

[ml21]

Did you try it, @ml21?

@pbek I haven’t tried devbox yet because I want to use as few additional utilities as possible. I simply specify the version (found on nixhub) of the nixpkgs that has the version of playwright I need.

For example, I need playwright v1.34:

{
  description = "dev env";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    nixpkgs-pw-v1_34.url = "github:NixOS/nixpkgs/dfcffbd74fd6f0419370d8240e445252a39f4d10";
  };

  outputs = { self, nixpkgs, nixpkgs-pw-v1_34 }: let
    system = "x86_64-linux";
    pkgs = import nixpkgs { inherit system; };
    pkgs-pw-v1_34 = import nixpkgs-pw-v1_34 { inherit system; };
  in {
      devShells.${system}.default = pkgs.mkShell {
      packages = [
        pkgs.nodejs_18
        pkgs.dotnet-sdk_8
        pkgs.process-compose
        pkgs-pw-v1_34.playwright
      ];

      PLAYWRIGHT_NODEJS_PATH = "${pkgs.nodejs}/bin/node";
      PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true;
      PLAYWRIGHT_BROWSERS_PATH="${pkgs-pw-v1_34.playwright-driver.browsers}";

      shellHook = ''
        echo "dev env started"
        fish && echo "exit dev env" && exit
      '';
    };
  };
}

[pbek]

@ml2, yes that's what I would have done too. Meanwhile, I tried devbox, works great too to pin the version.

[tobiasBora]

So another solution is to use nix-ld that is available by default in nixos using something like this in your configuration.nix:

  # Automatically creates a loader in /lib/* to avoid patching stuff
  # To disable it temporarily use
  # unset NIX_LD
  programs.nix-ld = {
    enable = true;
    libraries = with pkgs; [
      # List by default
      zlib
      zstd
      stdenv.cc.cc
      curl
      openssl
      attr
      libssh
      bzip2
      libxml2
      acl
      libsodium
      util-linux
      xz
      systemd

      # My own additions
            xorg.libXcomposite
      xorg.libXtst
      xorg.libXrandr
      xorg.libXext
      xorg.libX11
      xorg.libXfixes
      libGL
      libva
      pipewire
      xorg.libxcb
      xorg.libXdamage
      xorg.libxshmfence
      xorg.libXxf86vm
      libelf

      # Required
      glib
      gtk2

      # Without these it silently fails
      xorg.libXinerama
      xorg.libXcursor
      xorg.libXrender
      xorg.libXScrnSaver
      xorg.libXi
      xorg.libSM
      xorg.libICE
      gnome2.GConf
      nspr
      nss
      cups
      libcap
      SDL2
      libusb1
      dbus-glib
      ffmpeg
      # Only libraries are needed from those two
      libudev0-shim

      # needed to run unity
      gtk3
      icu
      libnotify
      gsettings-desktop-schemas
      # https://github.com/NixOS/nixpkgs/issues/72282
      # https://github.com/NixOS/nixpkgs/blob/2e87260fafdd3d18aa1719246fd704b35e55b0f2/pkgs/applications/misc/joplin-desktop/default.nix#L16
      # log in /home/leo/.config/unity3d/Editor.log
      # it will segfault when opening files if you don’t do:
      # export XDG_DATA_DIRS=/nix/store/0nfsywbk0qml4faa7sk3sdfmbd85b7ra-gsettings-desktop-schemas-43.0/share/gsettings-schemas/gsettings-desktop-schemas-43.0:/nix/store/rkscn1raa3x850zq7jp9q3j5ghcf6zi2-gtk+3-3.24.35/share/gsettings-schemas/gtk+3-3.24.35/:$XDG_DATA_DIRS
      # other issue: (Unity:377230): GLib-GIO-CRITICAL **: 21:09:04.706: g_dbus_proxy_call_sync_internal: assertion 'G_IS_DBUS_PROXY (proxy)' failed

      # Verified games requirements
      xorg.libXt
      xorg.libXmu
      libogg
      libvorbis
      SDL
      SDL2_image
      glew110
      libidn
      tbb

      # Other things from runtime
      flac
      freeglut
      libjpeg
      libpng
      libpng12
      libsamplerate
      libmikmod
      libtheora
      libtiff
      pixman
      speex
      SDL_image
      SDL_ttf
      SDL_mixer
      SDL2_ttf
      SDL2_mixer
      libappindicator-gtk2
      libdbusmenu-gtk2
      libindicator-gtk2
      libcaca
      libcanberra
      libgcrypt
      libvpx
      librsvg
      xorg.libXft
      libvdpau
      # ...
      # Some more libraries that I needed to run programs
      gnome2.pango
      cairo
      atk
      gdk-pixbuf
      fontconfig
      freetype
      dbus
      alsaLib
      expat
      # Needed for electron
      libdrm
      mesa
      libxkbcommon
      # Needed to run, via virtualenv + pip, matplotlib & tikzplotlib
      stdenv.cc.cc.lib # to provide libstdc++.so.6
      pkgs.gcc-unwrapped.lib # maybe only the first one needed

      # needed to run appimages
      fuse # needed for musescore 4.2.1 appimage
      e2fsprogs # needed for musescore 4.2.1 appimage
    ];
  };  

and then to run

$ export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$NIX_LD_LIBRARY_PATH"
$ npx playwright test --ui  

this allows you to use playwright like in any non-nixos system.

[rjpcasalino]

@tobiasBora you just blew my mind; I get some missing stuff for webkit but not a real issue for me