search

PCextreme / cloudstack-php

Native PHP Cloudstack client
MIT License
10 stars 9 forks source link

PROPOSAL: e2e testing #23

Closed FrontEndCoffee closed 5 years ago

FrontEndCoffee commented 6 years ago

Proposal

The client is currently only covered by unit tests. This is awesome, but I think we should do more.

I propose we implement end-to-end tests. And I have an idea on how to do it.

The goal of these tests is to test all possible commands through the client.

Implementation

Note: The e2e tests SHOULD NOT count towards the code coverage statistic, this should be reserved for unit tests. This can be done using the @codeCoverageIgnore annotation.

My proposed implementation would add a single test class that is called e2eTest.php. This class would consist of a test method, and a data provider method.

testEndToEndCase($args...)

Instantiate the client like normal, the only difference from a real world example is that the sendRequest method gets mocked to return a static response, and assert the provided URL and request parameters. After that the eventual return value of the client is asserted for it's value.

e2eProvider()

PHPUnit data provider that has the following structure:

[
  [
    'command': <string>,
    'parameters': <array>,
    'expectedURL': <string>,
    'cloudstackResponse': <string>,
    'clientResponse': <array|string>
  ],
  ...
]
BackEndTea commented 6 years ago

For e2e testing, consider https://github.com/Behat/Behat

FrontEndCoffee commented 5 years ago

Working on this as a proof of concept

https://github.com/FrontEndCoffee/cloudstack-php/tree/e2e

kevindierkx commented 5 years ago

I like the concept of end to end testing, however for a PHP component that doesn't actually provide 'an end', like a front-end or a cli environment (excluding the console command for the api list), I think it's rather redundant to write BDT's.

Is there a scenario where basic UnitTests, which we already provide, wouldn't cover what an e2e test would?

kevindierkx commented 5 years ago

As discussed internally, we won't add E2E tests to this package at the moment.