Breaking change moving to Percy-Capybara v4

[bjeanes]

With <v4, Percy worked with remote Selenium endpoints. We don't drive a local Chrome/FF browser but instead a remote one.

Now, Percy based on the agent expects a local Chrome and chromedriver to do snapshotting.

Is there anyway to upgrade without adding a local Chrome to our CI containers?

Is #137 building towards something like this?

[djones]

Hi @bjeanes,

Thanks for the note.

Is there anyway to upgrade without adding a local Chrome to our CI containers?

There is not.

Percy Agent depends on Puppeteer, which in turn does download a guaranteed compatible version of Chromium. And so that's what you're seeing. I believe you can supply a path to Chrome directly if you don't want to use the bundled one.

Chromedriver is not involved. Puppeteer speaks directly to Chromium using the Chrome Remote Debugging Protocol.

I sense some hesitation to including a browser in CI. Could you elaborate on that? I'd like to better understand your CI environment. We don't usually find this to be a problem so I'm curious what your use case is.

P.S. regardless of the outcome of this discussion, we plan to continue to support v3 for quite a long time so don't feel too pressured to upgrade if you're unable to do so at this time 🙂

[bjeanes]

Our hesitation is really just that our CI is pretty resource intensive and we intentionally run our Chrome elsewhere over Selenium. Adding a local Chromium might require us to change our base Docker image and add other dependencies we don't currently use, like libX11

/app/node_modules/puppeteer/.local-chromium/linux-641577/chrome-linux/chrome: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory
  | TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md
  | at onClose (~/node_modules/puppeteer/lib/Launcher.js:342:14)
  | at Interface.helper.addEventListener (~/node_modules/puppeteer/lib/Launcher.js:331:50)
  | error Command failed with exit code 1.

[bjeanes]

To add to this, here is the full list of dependencies I'd need to add to our build just to run puppeteer:

  at-spi2-core chromium fonts-liberation i965-va-driver libaacs0 libasound2 libasound2-data libasyncns0 libatspi2.0-0 libauthen-sasl-perl libavcodec57 libavformat57 libavutil55 libbdplus0
  libbluray1 libchromaprint1 libcrystalhd3 libdrm-amdgpu1 libdrm-intel1 libdrm-nouveau2 libdrm-radeon1 libdrm2 libencode-locale-perl libfile-basedir-perl libfile-desktopentry-perl
  libfile-listing-perl libfile-mimeinfo-perl libflac8 libfont-afm-perl libfontenc1 libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa libgme0 libgsm1 libhtml-form-perl libhtml-format-perl
  libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl libhttp-message-perl libhttp-negotiate-perl libio-html-perl
  libio-socket-ssl-perl libipc-system-simple-perl libllvm3.9 liblwp-mediatypes-perl liblwp-protocol-https-perl libmailtools-perl libminizip1 libmp3lame0 libmpg123-0 libnet-dbus-perl
  libnet-http-perl libnet-smtp-ssl-perl libnet-ssleay-perl libnspr4 libnss3 libnuma1 libogg0 libopenmpt0 libopus0 libpci3 libpciaccess0 libpulse0 libre2-3 libsensors4 libshine3 libsnappy1v5
  libsndfile1 libsoxr0 libspeex1 libssh-gcrypt-4 libswresample2 libtext-iconv-perl libtheora0 libtie-ixhash-perl libtimedate-perl libtwolame0 libtxc-dxtn-s2tc liburi-perl libva-drm1 libva-x11-1
  libva1 libvdpau-va-gl1 libvdpau1 libvorbis0a libvorbisenc2 libvorbisfile3 libvpx4 libwavpack1 libwrap0 libwww-perl libwww-robotrules-perl libx11-protocol-perl libx11-xcb1 libx264-148
  libx265-95 libxaw7 libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0 libxcb-present0 libxcb-shape0 libxcb-sync1 libxft2 libxml-parser-perl libxml-twig-perl libxml-xpathengine-perl libxmu6 libxmuu1
  libxshmfence1 libxss1 libxtst6 libxv1 libxvidcore4 libxxf86dga1 libxxf86vm1 libzvbi-common libzvbi0 mesa-va-drivers mesa-vdpau-drivers perl-openssl-defaults tcpd va-driver-all vdpau-driver-all
  x11-utils x11-xserver-utils xdg-utils

That grows our Docker image by almost 500 MB!

[djones]

@bjeanes which docker image is your CI runs built upon?

[ticky]

We at @buildkite are in the same boat. We need to move to v4 in order to get much out of Percy because we’ve discovered that v3 doesn’t support half the stylesheets in our app (we use styled-components in much of our modern code, which writes style properties to a style tag without writing the stylesheet text).

Our CI setup is already orchestrated such that we are bringing in a separate selenium/standalone-chrome Docker service to run our Capybara tests in. Adding a custom Chromium-including image simply to enable Percy is just further support and maintenance burden which we’ve currently alleviated by using this third-party maintained Selenium image in our spec suite.

Because we don’t have a good solution to this we’re now blocked on decisions around the styling of some features because we know they will break our Percy diffs, but not because of any real-world problem.

[bjeanes]

@djones a few different images (usually slim variants of base language images like ruby and node) in a Docker compose arrangement, which is part of our reasons to factor out Selenium service, in addition to precisely the things @ticky has enumerated.

[bjeanes]

@djones has there been any progression since the initial v4 release?

@grrowl suggested that maybe there would be a way to spin up the v4 agent on a different image in our docker-compose setup and the Ruby gem could connect to a non-local agent? That way the agent can be co-located with its dependencies without affecting our leaner (non-Node, non-X11) application image?

[ticky]

@bjeanes did you ever find a viable workaround for this? We’ve just been stuck with the earlier agent, which we have had to bend over backwards to keep working.

[Robdel12]

Hey y'all! We have this doc for how to skip the download and provide a path to an existing Chrome browser: https://docs.percy.io/docs/skipping-puppeteer-chromium-download There aren't any plans in the near future for removing Puppeteer from the new SDKs. We have kicked around an idea for providing a way to connect to a remote browser using CDP, but that's a ways off from being spiked.

If you could figure out a way to tunnel http://localhost:5338 from one docker image running @percy/agent to the docker image running the tests, it should work. Basically we take a DOM snapshot from the test browser and POST' it to localhost:5338/percy/snapshot where @percy/agent does asset discovery (with Puppeteer) on the DOM snapshot.

From the Capybara SDK perspective, it doesn't care how localhost:5338 is started, just as long as it can accept DOM snapshots. In theory you could use percy start to start the server, and make it available to your test suite (which will then POST the DOM and everything works 🎉 ).

I'm going to close this issue, but feel free to continue to comment 👍

[bjeanes]

@ticky no I did not. I stayed on v3 and started looking for alternative stacks for image diffing (until I left that job)