Integrate regression testing

wttech / gradle-aem-multi

Example Multi-Module AEM application built by Gradle Build System

https://tech.cognifide.com/tag/gradle-aem-plugin

Apache License 2.0

31 stars 18 forks source link

Integrate regression testing #67

Open pun-ky opened 5 years ago

pun-ky commented 5 years ago

Using: https://github.com/BBC-News/wraith Due to requirement about having ImageMagick installed on machine, it should be deployed on Docker via GAP environment plugin: http://bbc-news.github.io/wraith/os-install.html https://hub.docker.com/r/bbcnews/wraith/dockerfile/

Alternatively consider using AET https://github.com/Skejven/aet-docker , but seems that it is just more heavy for local dev machine (but better for production :) / but it is not the case here)

mmajchrzak commented 5 years ago

I would opt for using AET for a number of reasons:

more comprehensive than Wraith, e.g. comparing various breakpoints, js+http errors, advanced control on image comparison
reusing tests across local and shared envs. If you go for Wraith on local and AET on shared envs, you'll end up with test duplication which is a really bad idea - nobody would like to maintain such a setup

I appreciate the fact that AET may be heavy on local machine. In this case I'd cooperate with AET team on how we can make the setup lighter rather than investing the effort in integrating Wraith.

pun-ky commented 5 years ago

@Skejven how about creating compact / portable / lightweight version of AET? ;)

malaskowski commented 5 years ago

Hi,

compact / portable / lightweight version of AET

This idea was in my mind for some time... The whole complication is that the most "heavy" in AET setup are browsers and database. While I can imagine getting rid of the latter (with some e.g. in-memory storage), then AET would rather be useless without access to the browser... What comes to my mind when we talk about AET-lite is using the host browser. This has some disadvantages though (like handling browser auto-update etc.), which might not be a case in the dev env.

mmajchrzak commented 5 years ago

So if we make the database aspect of AET lighter, then AET should be comparable to Wraith, right? @pun-ky have you validated how heavy the Wraith is? It also uses headless browsers (selenium+Chromedriver)...

malaskowski commented 5 years ago

Well, it looks that Wraith is just comparing images. AET has quite a wider range of possibilities :)

pun-ky commented 5 years ago

Just looked for alternatives. No. I just want to check screenshots, really (the simpler solution the better, the plan is to check it for each build, not ocassionaly so the time is crucial). All other stuff probably could be handled by webhint and lighthouse.

pun-ky commented 5 years ago

Another option could be to try with https://github.com/palmerhq/cypress-image-snapshot as of there are already plans to check also cypress.io in case of functional tests