Closed VaasRamsay closed 3 years ago
@SamBreaksThings we don't support Nix at the moment and this is the first request we've received regarding it so far.
So until there's a huge demand for Nix support, this will be low-pri for us. Is there something easy we can do to enable you use Playwright on Nix?
Is it possible to make the linked workaround configurable: https://github.com/microsoft/playwright/blob/a9faa9c941fa0b0383d68dff2fb7ef825bd258ea/src/server/validateDependencies.ts#L247-L250?
Note the commit message of https://github.com/aslushnikov/playwright/commit/aa3019bc8225abd463602481e3362fc61ea33b40
To the best of my knowledge,
ldconfig
is always located at/sbin
.
This is one case where it's not located there, and loading it from there leads to broken behaviour.
Is it possible to make the linked workaround configurable:
maybe via an env var await spawnAsync(process.env.LDCONFIG_PATH || '/sbin/ldconfig', ['-p'], {});
? Or if that is an unwanted API it would also be thinkable to make it a fallback from the generic ldconfig
(w/o qualified path, e.g. command -v ldconfig
or which ldconfig
) and if that one is not found to fall back to /sbin/ldconfig
. Or if we want to make the speed sacrifice for more exotic users of this package if could be in the catch block of the /sbin/ldconfig
spawn and introspect the error and retry with ldconfig
from path?
Alternatively an option to entirely disable validating dependencies would work even better: I'm noticing ldconfig
is only used to provide an error message suggesting how to install missing dependencies.
@uri-canva @joscha Is this the only thing that breaks Playwright on Nix? In this case, would symlinking the right ldconfig to /sbin/ldconfig
solve it on your end?
A system can have multiple nix profiles. Which one is used is dependent on the environment started. These virtualizations only affect the PATH
(and a few other things like env) but not the file system, so changing a symlink is not really idiomatic as you would suddenly bind the static filesystem to a specific, arbitrary nix profile.
On CI (where we intend to run this) it would mean that the symlink would point to a random profile or possibly nothing at all.
I understand that an env var driving this might be an API we don't want to introduce but what do you think about falling back from the entry in PATH
to the hardcoded location or skipping use of ldconfig altogether like @uri-canva suggested?
@joscha if this is the only thing that doesn't work on Nix - then sure, we can come up with something as long as it's non-intrusive for the codebase & API.
But is this the only thing that prevents you from running on NIX? For example, I'll be surprised if our browser builds launch on NIX. Have you tried running them?
The browsers require a wrapper around them to launch on nix: https://github.com/ludios/nixos-playwright
The browsers require a wrapper around them to launch on nix: https://github.com/ludios/nixos-playwright
@uri-canva impressive!
So to double-check: the ldconfig
is the last thing that stops you folks from using playwright on nix?
Would you be open to send a PR that detects *NIX system and disables launch doctor there? I cannot guarantee we land it, but we'll discuss it. I'd appreciate if it's as non-invasive as possible.
P.S. I apologize for all the questions - I have zero experience with NIX systems, and playing with it in docker was painful - there's even no bash 🤷♂️
We can make it agnostic to nix: look if ldconfig
is on PATH
, if not try /sbin/ldconfig
, if that doesn't exist either fail?
This sounds good to me!
I have made a PR that allows disabling host requirements validation, please have a look https://github.com/microsoft/playwright/pull/5806.
Since the PR for skipping the host validation got merged and there won't be official supported added we will close it for now, feel free to reopen / upvote it when there is a higher demand.
@olebedev , would you mind sharing a repo link (or gist) for the Nix derivation you use to wrap playwright's browser binaries as referenced in #5806? Or are you using the one linked above?
Apologies for commenting on an old issue; I considered sending an email, but figured asking in a public forum might help others with the same question.
@olebedev , would you mind sharing a repo link (or gist) for the Nix derivation you use to wrap playwright's browser binaries as referenced in #5806? Or are you using the one linked above?
@kylegentle, no, I use my own-written Nix derivation. It's not just a code snippet so I need to go through the OSS request process within the company I work for, to make it public. I will keep you posted here.
Playwright package is now available in unstable channel.
I got it working with nix flake by using following flake.nix
and then running nix develop
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
};
outputs = { self, nixpkgs }: with nixpkgs.legacyPackages.x86_64-linux; {
devShell.x86_64-linux = mkShell {
buildInputs = [
playwright
playwright.browsers
];
shellHook = ''
export PLAYWRIGHT_BROWSERS_PATH=${playwright.browsers}
export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=true
'';
};
};
}
playwright.browsers
I have 22.11 with playwright installed through config... that may work for python, but packages that depend on playwright like Node Slidev still don't work and you'll get the dreaded message:
Host system is missing dependencies to run browsers. ║
║ Missing libraries: ║
║ libgobject-2.0.so.0 ║
║ libglib-2.0.so.0 ║
║ libnss3.so ║
║ libnssutil3.so ║
║ libsmime3.so ║
║ libnspr4.so ║
║ libatk-1.0.so.0 ║
║ libatk-bridge-2.0.so.0 ║
║ libcups.so.2 ║
║ libgio-2.0.so.0 ║
║ libdrm.so.2 ║
║ libdbus-1.so.3 ║
║ libatspi.so.0 ║
║ libX11.so.6 ║
║ libXcomposite.so.1 ║
║ libXdamage.so.1 ║
║ libXext.so.6 ║
║ libXfixes.so.3 ║
║ libXrandr.so.2 ║
║ libgbm.so.1 ║
║ libexpat.so.1 ║
║ libxcb.so.1 ║
║ libxkbcommon.so.0 ║
║ libpango-1.0.so.0 ║
║ libcairo.so.2 ║
║ libasound.so.2
I get the same error message...
installed playwright
via NixOS /etc/nixos/configuration.nix
in users.users.<me>.packages = with pkgs; [playwright];
playwright install
does download the 3 different browser drivers, with a warning BEWARE: your OS is not officially supported by Playwright; downloading fallback build.
And finally, playwright open
or any command like that causes this error (see above^).
I am going to link this issue on the nixpkgs repo to see if we can get some help from the NixOS community...
Calling the heroes in: @techknowlogick @yrd @SuperSandro2000
playwright install
does download the 3 different browser drivers, with a warningBEWARE: your OS is not officially supported by Playwright; downloading fallback build.
The Playwright package splits the browser bundle depending on the platform. For Linux, playwright.browsers
is a derivation that emulates what you get when running playwright install
, but with the Nixpkgs version of each browser. As @aabccd021 mentioned, you will need to set PLAYWRIGHT_BROWSERS_PATH
to this derivation, instead of having Playwright download the precompiled browsers. Try something like this:
export PLAYWRIGHT_BROWSERS_PATH=$(nix build --print-out-paths nixpkgs#playwright.browsers)
nix shell nixpkgs#playwright
playwright open nixos.org
(Add --no-link
to the nix build
step if you don't wan't the result
symlink to the browsers derivation). I just tested this using NixOS/nixpkgs@5a350a8f31bb7ef0c6e79aea3795a890cf7743d4 and it seems to work:
If you're on Darwin, playwright.browsers
will be the same bundle you get when running playwright install
(the pre-built binaries by the Playwright team, including their patches), just as an FOD so we get the reproducibility. On Mac, using Playwright's install script as @adam248 described should work as well, but playwright.browsers
in combination with the environment variable will keep everything in the Nix store.
One more note when using our browser bundle on Linux instead of the Playwright-provided one: this does mean that you won't be getting any Playwright-specific browser patches. Maybe someone from the Playwright team could weigh in on the implications here?
Applying a 3000 or 23000 line patch is guaranteed to break at some point. Unless those patches get split up into smaller ones, we cannot maintainable apply them to a source build unless we fetch the same exact version.
I just don't know if there are any Playwright Features we're currently missing out on by using the regular browser builds… But I do agree with @SuperSandro2000 that it's infeasible for us to include all patches. Maybe there are some parts which could be interesting to cherry-pick, but I'm not versed enough in Playwright's internals to say anything about that.
Currently, this shell.nix
is working for me:
{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
# nativeBuildInputs is usually what you want -- tools you need to run
nativeBuildInputs = with pkgs; [
playwright-driver.browsers
];
shellHook = ''
export PLAYWRIGHT_BROWSERS_PATH=${pkgs.playwright-driver.browsers}
export PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS=true
'';
}
@pbek It works also for me with node package @playwright/test
version 1.40.1 with the same shell.nix.
I updated to 1.41.1 and now I can't launch playwright : Error No chromium-based browser found on the system.
I've downgraded for now...
I already added the version matching in the nix wiki last week, because I had a similar issue. 😅
Sorry, if my questions are stupid - I'm newbie in nix.
I already added the version matching in the nix wiki last week, because I had a similar issue. 😅
I tried to find it in wiki https://nixos.wiki/wiki/Playwright , but no version matching there. Where can I find this version matching?
And one more question please - in wiki:
Note: Keep in mind that you need to use the same version of playwright in your node playwright project as in your nixpkgs, or else playwright will try to use browsers versions that aren't installed!
How can I specify playwright\playwright-driver.browsers version in my flake.nix?
For example, if my project uses playwright version 1.41 how can I specify right playwright\playwright-driver.browsers nixpkgs versions?
Here is my test flake with dev env
{
description = "dev env";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
};
outputs = { self, nixpkgs }: let
system = "x86_64-linux";
pkgs = import nixpkgs { inherit system; };
in {
devShells.${system}.default = pkgs.mkShell {
packages = with pkgs; [
nodejs
dotnet-sdk_8
playwright
];
PLAYWRIGHT_NODEJS_PATH = "${pkgs.nodejs}/bin/node";
PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true;
PLAYWRIGHT_BROWSERS_PATH="${pkgs.playwright-driver.browsers}";
shellHook = ''
echo "dev env started"
fish && echo "exit dev env" && exit
'';
};
};
}
Note: Keep in mind that you need to use the same version of playwright in your node playwright project as in your nixpkgs, or else playwright will try to use browsers versions that aren't installed!
I was talking about that note. 😉
I was looking at https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=playwright-driver, noting the version. Then updating my flake. Then installing exact that version with npm too.
Then installing exact that version with npm too.
What if I can't change the version in the project (dotnet in my case, not node)?
In this case, I need to find and specify a suitable version of playwright from nixpkgs.
How could I do that?
—- Updated: I think I found a way to find old versions: https://www.nixhub.io/packages/playwright
Anyway - @pbek thank you, your posts here and on discourse very useful for me
Oh, that's nice. I didn't know that service! And I didn't know "devbox". Did you try it, @ml21? Does it create a flake for your project to pin the version? Because that's what I would have done. Pin the nix version of playwright (by searching the github nixpkgs repository for the commit hash of exactly the version that I need) with a nix flake.
Did you try it, @ml21?
@pbek I haven’t tried devbox yet because I want to use as few additional utilities as possible. I simply specify the version (found on nixhub) of the nixpkgs that has the version of playwright I need.
For example, I need playwright v1.34:
{
description = "dev env";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
nixpkgs-pw-v1_34.url = "github:NixOS/nixpkgs/dfcffbd74fd6f0419370d8240e445252a39f4d10";
};
outputs = { self, nixpkgs, nixpkgs-pw-v1_34 }: let
system = "x86_64-linux";
pkgs = import nixpkgs { inherit system; };
pkgs-pw-v1_34 = import nixpkgs-pw-v1_34 { inherit system; };
in {
devShells.${system}.default = pkgs.mkShell {
packages = [
pkgs.nodejs_18
pkgs.dotnet-sdk_8
pkgs.process-compose
pkgs-pw-v1_34.playwright
];
PLAYWRIGHT_NODEJS_PATH = "${pkgs.nodejs}/bin/node";
PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS = true;
PLAYWRIGHT_BROWSERS_PATH="${pkgs-pw-v1_34.playwright-driver.browsers}";
shellHook = ''
echo "dev env started"
fish && echo "exit dev env" && exit
'';
};
};
}
@ml2, yes that's what I would have done too. Meanwhile, I tried devbox, works great too to pin the version.
So another solution is to use nix-ld
that is available by default in nixos using something like this in your configuration.nix
:
# Automatically creates a loader in /lib/* to avoid patching stuff
# To disable it temporarily use
# unset NIX_LD
programs.nix-ld = {
enable = true;
libraries = with pkgs; [
# List by default
zlib
zstd
stdenv.cc.cc
curl
openssl
attr
libssh
bzip2
libxml2
acl
libsodium
util-linux
xz
systemd
# My own additions
xorg.libXcomposite
xorg.libXtst
xorg.libXrandr
xorg.libXext
xorg.libX11
xorg.libXfixes
libGL
libva
pipewire
xorg.libxcb
xorg.libXdamage
xorg.libxshmfence
xorg.libXxf86vm
libelf
# Required
glib
gtk2
# Without these it silently fails
xorg.libXinerama
xorg.libXcursor
xorg.libXrender
xorg.libXScrnSaver
xorg.libXi
xorg.libSM
xorg.libICE
gnome2.GConf
nspr
nss
cups
libcap
SDL2
libusb1
dbus-glib
ffmpeg
# Only libraries are needed from those two
libudev0-shim
# needed to run unity
gtk3
icu
libnotify
gsettings-desktop-schemas
# https://github.com/NixOS/nixpkgs/issues/72282
# https://github.com/NixOS/nixpkgs/blob/2e87260fafdd3d18aa1719246fd704b35e55b0f2/pkgs/applications/misc/joplin-desktop/default.nix#L16
# log in /home/leo/.config/unity3d/Editor.log
# it will segfault when opening files if you don’t do:
# export XDG_DATA_DIRS=/nix/store/0nfsywbk0qml4faa7sk3sdfmbd85b7ra-gsettings-desktop-schemas-43.0/share/gsettings-schemas/gsettings-desktop-schemas-43.0:/nix/store/rkscn1raa3x850zq7jp9q3j5ghcf6zi2-gtk+3-3.24.35/share/gsettings-schemas/gtk+3-3.24.35/:$XDG_DATA_DIRS
# other issue: (Unity:377230): GLib-GIO-CRITICAL **: 21:09:04.706: g_dbus_proxy_call_sync_internal: assertion 'G_IS_DBUS_PROXY (proxy)' failed
# Verified games requirements
xorg.libXt
xorg.libXmu
libogg
libvorbis
SDL
SDL2_image
glew110
libidn
tbb
# Other things from runtime
flac
freeglut
libjpeg
libpng
libpng12
libsamplerate
libmikmod
libtheora
libtiff
pixman
speex
SDL_image
SDL_ttf
SDL_mixer
SDL2_ttf
SDL2_mixer
libappindicator-gtk2
libdbusmenu-gtk2
libindicator-gtk2
libcaca
libcanberra
libgcrypt
libvpx
librsvg
xorg.libXft
libvdpau
# ...
# Some more libraries that I needed to run programs
gnome2.pango
cairo
atk
gdk-pixbuf
fontconfig
freetype
dbus
alsaLib
expat
# Needed for electron
libdrm
mesa
libxkbcommon
# Needed to run, via virtualenv + pip, matplotlib & tikzplotlib
stdenv.cc.cc.lib # to provide libstdc++.so.6
pkgs.gcc-unwrapped.lib # maybe only the first one needed
# needed to run appimages
fuse # needed for musescore 4.2.1 appimage
e2fsprogs # needed for musescore 4.2.1 appimage
];
};
and then to run
$ export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$NIX_LD_LIBRARY_PATH"
$ npx playwright test --ui
this allows you to use playwright like in any non-nixos system.
@tobiasBora you just blew my mind; I get some missing stuff for webkit
but not a real issue for me
Context:
Describe the bug
Running Playwright in Nix Shell throws
missing dependencies
error. Even after explicitly adding the missing dependencies suggested by "launch doctor", the same error is thrown.Probably, I'm hitting this error from here. Nix has it's own
ldconfig
and it's not found insbin
.