google / site-kit-wp

Site Kit is a one-stop solution for WordPress users to use everything Google has to offer to make them successful on the web.
https://sitekit.withgoogle.com
Apache License 2.0
1.23k stars 279 forks source link

Fix issue where VRT test fails in CI when launching Chromium #9211

Closed techanvil closed 1 week ago

techanvil commented 3 weeks ago

Bug Description

Unfortunately the experimental fix for our VRT workflow flakiness applied via https://github.com/google/site-kit-wp/issues/7965 doesn't seem to have improved its reliability and the workflow is still failing at a high rate during peak US/EU working hours.

VRT runs in CI are still intermittently failing before all the screenshots are captured, with the following error:

      COMMAND | Command "test" ended with an error after [542.215s]
      COMMAND | Error: Failed to launch the browser process!

Steps to reproduce

This error has been seen with a couple of permutations:

image

image

image

See this Slack thread for related discussion on the topic, including some suggested avenues for investigation:


Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

Note, additional issues should be raised if needed to follow up on separate avenues of investigation.

Implementation Brief

After substantial investigation, it seems this may be due to resource limitations when running the Docker container. See https://github.com/garris/BackstopJS/issues/537#issuecomment-339710797

Test Coverage

QA Brief

Changelog entry

tofumatt commented 3 weeks ago

Moving directly to Approval given no QA is required here.