search

acquia / blt

Acquia's toolset for automating Drupal 8 and 9 development, testing, and deployment.
https://docs.acquia.com/blt/
GNU General Public License v2.0
443 stars 396 forks source link

BLT & Visual Regression Strategy #1072

Closed justinlevi closed 7 years ago

justinlevi commented 7 years ago

We currently have phpUnit and behat tests integrated into the BLT project. I'd love to see a default recommended visual regression strategy also included with BLT. Selfishly, and I think it would be good for Acquia too, I'd like to see Acquia Cloud and ACSF environments addressed first. I think having a starting point for integrating Visual Regression tests with BLT could be really helpful.

Pantheon has a good blog post about using Wraith https://pantheon.io/docs/guides/visual-diff-with-wraith/

About a year ago I investigated Webdriverio and webdrivercss and casperjs but for some reason I feel like there were some conflicts/issues there... https://github.com/webdriverio/webdrivercss

Integration w/ Browsershots might also be advantageous.

The motivation behind an "official" approach is that the entire space seems very unapproachable for most teams. Creating a spike, even if it's not the final approach, could create a launching point for some really interesting and useful conversations.

As I see it, the acceptance criteria for a feature like this would include the ability for the developer to easily run visual regression tests locally prior to committing a PR. During the automated build and deployment process it would be great to have these tests run as well.

I realize that there are quite a few different workflows and strategies for visual regression testing but having a baseline would be really helpful IMHO.

Here's an older thread I started over on the Drupal-VM github issue queue as well: https://github.com/geerlingguy/drupal-vm/issues/421

grasmash commented 7 years ago

Paging @jenter and @sarahjean :)

justinlevi commented 7 years ago

I am more than happy to work on this as well, would just like to get some thoughts before I dive in.

sarahjean commented 7 years ago

My main complaint with Wraith is the ruby dependency. I think it'd be better to look into a setup with PhantomCSS: https://www.phase2technology.com/blog/css-testing-with-phantomcss-phantomjs-casperjs-and-grunt/

justinlevi commented 7 years ago

Thoughts on BackstopJS? https://github.com/garris/BackstopJS

chrowe commented 7 years ago

That looks like a good option to me. Others are using it

It says "BackstopJS may be just the thing if you develop custom WordPress, Drupal or other CMS templates. Tested on OSX."

grasmash commented 7 years ago

I think we should look at adding this to Cog rather than BLT.

Cog already manages dependencies with NPM. BLT does not, and it seems like a heavyweight addition. It could also conflict with node based requirements for project themes.

We should then more formally integrate Cog with BLT.

grasmash commented 7 years ago

I'm going to close this. Happy to support integration with Cog, but not willing to add any NPM dependencies to BLT itself at this time.