Open rickmanelius opened 6 years ago
As part of our focus on foundational items in milestone v0.3, I think it would be wise to review how we're currently doing integration tests and re-evaluate to see if we need to take a different approach to ensure proper coverage of all functionality through v0.2 and that it will persist moving forward.
There are probably 3 specific areas that I think need implementation in terms of tests.
Current Test Strategy is Mocha unit tests that leverage fixtures and mocks to simulate output from modules with controlled input. What is lacking is integration tests that actually exercise an actual CLI instance, and browser-level click testing a-la selenium.
Proposed solution:
Testing with Spectron allows us to test the electron specific features as well as verify the presentational flow (clicks, modals, spinner appearance, etc) so that's highly preferred. The unit tests will ensure the modules are behaving as expected, the integration will verify the contract between CLI and ddev-shell wrapper are good and have not changed/broken with later builds.
One problem that after hours of searching I still have no answer to: how the heck can we test sudo access via a test? Though this won't be an issue any longer if/when we get the hostname issue resolved.
There are a couple of levels of integration we need to deal with:
They can be separate approaches.
On sudo: It's possible we don't have to test it. If the test setup creates the needed hostnames in /etc/hosts, sudo will never be needed. We would rather test sudo usage of course. But ddev start
would run in this situation without needing any privileges.
TODO: update this issue with the agreed, initial testing strategy.
adding @mafriend - this is an epic that contains (some) of the current priorities around testing
What happened (or feature request):
Updated A/C - 11/27/18
Given the amount of user interaction on the UI, we probably need a more robust approach for integration testing with sample data.
This epic includes: