e2e testing tool

[igmoweb]

I'm currently experimenting with e2e testing in a project. It started with some sort of automation that let me do some boring and long tasks quickly but I ended up using Jest and WordPress e2e utils.

I'd like to propose the introduction of a dev tools command (something that Rob has already thought about) in Altis that lets you run e2e tests (if there's any) with composer dev-tools e2e or similar.

There are a few considerations before working on this feature:

• The tests must be done in a different instance of Local Server. It's better not to use the same instance where you usually work as it could fill the DB easily and writing tests can be very slow if you have a production DB locally. Additionally, the tests can throw errors due to conflicts with the usual instance.
• In order to create a different instance, Local Server might need to accept an extra parameter like --name=e2e
• Running the tests and activating all the plugins, creating users/posts needed to run those tests can take a lot of time so having an initial DB would be very handy. This can be imported via WP CLI when the tests start running. It's up to the developers to provide that .sql file.
• Everything should play nicely with Travis. We may want to integrate some Travis images in Docker Hub but I haven't done any research on this and I'm not sure if we'll really need it.
• It can work similarly to PHPUnit, searching in similar folders to find test files.
• It can act as a wrapper for [@wordpress/scripts npm package](https://www.npmjs.com/package/@wordpress/scripts). This set of scripts offers commands to launch the e2e tests along with [@wordpress/e2e-test-utils package](https://www.npmjs.com/package/@wordpress/e2e-test-utils).
• It could provide an extensible Jest/bootstrap config. WordPress Scripts also provides one in @wordpress/scripts/config/jest-e2e.config. Altis Jest config can extend that one and call a bootstrap.js defined by default.

I've created a little Gist that can serve as a guide of what files are needed to run the e2e scripts: https://gist.github.com/igmoweb/1d64f820a45b61036022b1270b5c24e3

[roborourke]

@shadyvb I'd forgotten about this one! Seems codeception covers it, not sure if it's possible to use the e2e-test-utils with that but I think we can close this out. Anything you think we should extract from here or capture elsewhere first?