Custom E2E Tests 2 - Local Test Environment

[Luc45]

This PR adds the capability of running disposable test environments using QIT CLI.

The new env commands:

 env
  env:up                [env:start] Starts a local test environment.
  env:down              [env:stop] Stops a local test environment.
  env:enter             Enter the PHP container of a running test environment.
  env:exec              Execute a command inside the PHP container of a running test environment.
  env:list              List running environments.
  env:restart           Restarts a local test environment.

You can spin up a configurable environment, too, choosing the WordPress, WooCommerce, PHP version, PHP extensions and whether to have Object Cache or not:

qit env:up --help                                                                                                                                                                                                                                   

Description:
  Creates a temporary local test environment that is completely ephemeral — no data is persisted. Every time you stop and restart the environment, it's like starting fresh.

Usage:
  env:up [options]
  env:start

Options:
      --wordpress_version[=WORDPRESS_VERSION]      (Optional) The WordPress version to use in the test. [possible values: 6.5-RC1, 6.1.5, 6.2.4, 6.3.3, 6.4.3, stable, rc] [default: "6.4.3"]
      --woocommerce_version[=WOOCOMMERCE_VERSION]  (Optional) The WooCommerce version to use in the test. [possible values: 8.7.0-rc.1, 8.6.0, 8.6.1, stable, rc] [default: "8.6.1"]
      --php_version[=PHP_VERSION]                  (Optional) The PHP version to use in the test. [possible values: 7.4, 8.0, 8.1, 8.2, 8.3] [default: "7.4"]
  -m, --volume[=VOLUME]                            (Optional) Additional volume mappings, eg: /home/mycomputer/my-plugin:/var/www/html/wp-content/plugins/my-plugin. (multiple values allowed)
  -p, --php-ext[=PHP-EXT]                          (Optional) PHP extension to install in the environment. (multiple values allowed)

This environment is the same minimalist, alpine-based environment we use in QIT CI, which we will use to run custom E2E tests locally. In the future, it can also be used to run all QIT tests locally.

It makes for a tighter integration between test development in a developer's machine and test running in CI.

We control the environment in the Manager, which means that we, from the Manager, control the test environments that all developers uses on their local machines, which gives us immense power and integration. It's essentially a managed testing environment that we control and we offer for the developers, essentially abstracting away all the complexity of managing an environment for them, while giving us total control over the test environment being used both in the developer's machine and in CI, for a truly integrated testing framework.

Testing Instructions

• Pull this branch on your QIT CLI
• Pull the Manager PR counterpart on your Manager
• qit switch local
• qit env:up
• Assert that it downloads the environment from the Manager and that it spins up a test environment
• Access the test environment URL that is given to you
• Repeat the steps, spinning up a different configuration
• Stop the environment, list the environment, restart the environment

[Luc45]

Some different environments where the local test environment has been tested:

• Linux Desktop (Docker Compose 2) - Cached :heavy_check_mark: 15 seconds
• Linux CI - Uncached :heavy_check_mark: 19~25 seconds
• MacOS CI :heavy_check_mark:
• Windows Docker Desktop Hyper-V (no WSL) :heavy_check_mark: (160 seconds~)
• Windows Docker Desktop using WSL 2 :heavy_check_mark: (130 seconds~)
• Windows Running Inside WSL - Uncached :heavy_check_mark: 27 seconds
• Windows Running Inside WSL - Cached :heavy_check_mark: 17 seconds
• MacOS Desktop (Docker Desktop) - 30~40s :heavy_check_mark:
• MacOS Desktop (OrbStack) - :hourglass:

[zhongruige]

I'm getting this PHP warning on the build from this branch (not on 0.4.1):

➜  qit-cli git:(24-02/custom-tests-2) ✗ ./qit backend:current
PHP Deprecated:  json_decode(): Passing null to parameter #1 ($json) of type string is deprecated in phar:////workspace/qit-cli/qit/src/Environment/EnvironmentMonitor.php on line 17

Noting that I am on PHP 8.3.3 and after this notice it still provides the environment as expected so it isn't impacting anything.

[Luc45]

Can you please run it using php qit-cli.php so that we can pinpoint the line?

[zhongruige]

Can you please run it using php qit-cli.php so that we can pinpoint the line?

If I try and run it in the src folder it gives me the following:

QIT_CLI\Commands\Environment\StartEnvironmentCommand
    => QIT_CLI\Environment\Environments\E2EEnvironment $e2e_environment
    => Error while making Symfony\Component\Filesystem\Filesystem $filesystem: parameter $filesystem is not optional and its type (Symfony\Component\Filesystem\Filesystem) cannot be resolved to a concrete class.

[Luc45]

I see, can you please run composer install and try again? (in the src directory)

[zhongruige]

I see, can you please run composer install and try again? (in the src directory)

Ack can't believe I missed that! That did it, the error is on line 21: src/Environment/EnvironmentMonitor.php on line 21