garris
commented
7 years ago I work at a very large global social media company. We build and manage multiple web apps for our members. We also have the same company-issued MacBooks with nearly identical environments. We have an implementation of BackstopJS on a single machine testing prod and another "shared" implementation in which the same set of BackstopJS reference files are shared via git alongside our development src.
In the first instance, the workflow is straightforward. On each canary deploy, a BackstopJS process checks for changes against our last canary deploy. The release engineer gets an email when any changes are found -- which, during the day, is expected. The release engineer will review and generally backstop approve any plausible changes. Any non-plausible changes are escalated up the chain.
In the second instance, each dev is responsible for running BackstopJS and attaching to their RB ticket. In that case ACL's are able to review all display changes (intended and in-intended) that are submitted with the RB code changes.
It is not a perfect system. There are still too many false positives in both cases stemming from known issues like asset latency and other environmental flux. That said, it catches a lot of real errors too. And overall it definitely improves quality. Like our other test systems -- we intend to improve it over time.
I am thinking about the effective workflows for BackstopJS.
Currently how I am using it with Jenkins jobs, which can be triggered via pull requests or manually and the test tasks are integrated on a live server.
What is your workflow ? Local, live ? CI tool ?