Unfortunately the experimental fix for our VRT workflow flakiness applied via https://github.com/google/site-kit-wp/issues/7965 doesn't seem to have improved its reliability and the workflow is still failing at a high rate during peak US/EU working hours.
VRT runs in CI are still intermittently failing before all the screenshots are captured, with the following error:
COMMAND | Command "test" ended with an error after [542.215s]
COMMAND | Error: Failed to launch the browser process!
Steps to reproduce
Watch for failing VRT test runs in CI.
They will intermittently fail with the above error, particularly so during peak US/EU hours.
Bug Description
Unfortunately the experimental fix for our VRT workflow flakiness applied via https://github.com/google/site-kit-wp/issues/7965 doesn't seem to have improved its reliability and the workflow is still failing at a high rate during peak US/EU working hours.
VRT runs in CI are still intermittently failing before all the screenshots are captured, with the following error:
Steps to reproduce
Screenshots
This error has been seen with a couple of permutations:
See this Slack thread for related discussion on the topic, including some suggested avenues for investigation:
--remote-debugging-pipe
) instead of a port (--remote-debugging-port
) so the WebSocket setup code where the error is raised doesn't run.Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance criteria
Note, additional issues should be raised if needed to follow up on separate avenues of investigation.
Implementation Brief
After substantial investigation, it seems this may be due to resource limitations when running the Docker container. See https://github.com/garris/BackstopJS/issues/537#issuecomment-339710797
Test Coverage
QA Brief
Changelog entry