thekid commented 1 year ago

Scope of Change

This RFC suggests to create a new testing library for the XP Framework. It will only work with baseless, single-instance test classes.

Rationale

Drop support for extends TestCase, supporting of which costs lots of code
Drop support for seldomly used features such as test timeouts, exotic Values variations or external provider methods
Simplify test actions, these should have not use exceptions for flow control
Integrate Assert::that() extended assertions to simplify commonly used test code

Functionality

In a nutshell, tests reside inside a class and are annotated with the Test attribute. Except for the changed namespace (see below) and the changed default output, this is basically identical to the current usage of the unittest library.

use test\{Assert, Test};

class CalculatorTest {

  #[Test]
  public function addition() {
    Assert::equals(2, (new Calculator())->add(1, 1));
  }
}

To run this test, use the test subcommand:

$ xp test CalculatorTest.class.php
> [PASS] CalculatorTest
  ✓ addition

Tests:       1 succeeded, 0 skipped, 0 failed
Memory used: 1556.36 kB (1610.49 kB peak)
Time taken:  0.001 seconds

Naming

The library will be called test, because it can not only run unittests. The top-level package used follows this, meaning the Assert class' fully qualified name is test.Assert.

Assert DSL

The following shorthand methods exist on the Assert class:

equals(mixed $expected, mixed $actual) - check two values are equal. Uses the util.Objects::equal() method internally, which allows overwriting object comparison.
notEquals(mixed $expected, mixed $actual) - opposite of above
true(mixed $actual) - check a given value is equal to the true boolean
false(mixed $actual) - check a given value is equal to the false boolean
null(mixed $actual) - check a given value is null
instance(string|lang.Type $expected, mixed $actual) - check a given value is an instance of the given type.

Extended assertions

The Assert::that() method starts an assertion chain:

Fluent assertions

Each of the following may be chained to Assert::that():

is(unittest.assert.Condition $condition) - Asserts a given condition matches
isNot(unittest.assert.Condition $condition) - Asserts a given condition does not match
isEqualTo(var $compare) - Asserts the value is equal to a given comparison
isNotEqualTo(var $compare) - Asserts the value is not equal to a given comparison
isNull() - Asserts the value is null
isTrue() - Asserts the value is true
isFalse() - Asserts the value is false
isInstanceOf(string|lang.Type $type) - Asserts the value is of a given type

Transformation

Transforming the value before comparison can make it easier to create the value to compare against. This can be achieved by chaining map() to Assert::that():

$records= $db->open('select ...');
$expected= ['one', 'two'];

// Before
$actual= [];
foreach ($records as $record) {
  $actual[]= $r['name'];
}
Assert::equals($expected, $actual);

// After
Assert::that($records)->mappedBy(fn($r) => $r['name'])->isEqualTo($expected);

Values and providers

#[Values] is a basic provider implementation. It can either return a static list of values as follows:

use test\{Assert, Test, Values};

class CalculatorTest {

  #[Test, Values([[0, 0], [1, 1], [-1, 1]])]
  public function addition($a, $b) {
    Assert::equals($a + $b, (new Calculator())->add($a, $b));
  }
}

...or invoke a method, as seen in this example:

use test\{Assert, Test, Values};

class CalculatorTest {

  private function operands(): iterable {
    yield [0, 0];
    yield [1, 1];
    yield [-1, 1];
  }

  #[Test, Values(from: 'operands')]
  public function addition($a, $b) {
    Assert::equals($a + $b, (new Calculator())->add($a, $b));
  }
}

Provider implementation example

use test\Provider;
use lang\reflection\Type;

class StartServer implements Provider {
  private $connection;

  /** Starts a new server */
  public function __construct(string $bind, ?int $port= null) {
    $port??= rand(1024, 65535);
    $this->connection= "Socket({$bind}:{$port})"; // TODO: Actual implementation ;)
  }

  /**
   * Returns values
   *
   * @param  Type $type
   * @param  ?object $instance
   * @return iterable
   */
  public function values($type, $instance= null) {
    return [$this->connection];
  }
}

Provider values are passed as method argument, just like #[Values], the previous only implementation.

use test\{Assert, Test};

class ServerTest {

  #[Test, StartServer('0.0.0.0', 8080)]
  public function connect($connection) {
    Assert::equals('Socket(0.0.0.0:8080)', $connection);
  }
}

Provider values are passed to the constructor and the connection can be used for all test cases. Note: The provider's values() method is invoked with $instance= null!

use test\{Assert, Test};

#[StartServer('0.0.0.0', 8080)]
class ServerTest {

  public function __construct(private $connection) { }

  #[Test]
  public function connect() {
    Assert::equals('Socket(0.0.0.0:8080)', $this->connection);
  }
}

Test prerequisites

Prerequisites can exist on a test class or a test method. Unlike in the old library, they do not require the Action annotation, but stand alone.

use test\{Assert, Test};
use test\verify\{Runtime, Condition};

#[Condition('class_exists(Calculator::class)')]
class CalculatorTest {

  #[Test, Runtime(php: '^8.1')]
  public function addition() {
    Assert::equals(2, (new Calculator())->add(1, 1));
  }
}

Security considerations

n/a

Speed impact

Same

Dependencies

https://github.com/xp-framework/reflection

Related documents

thekid commented 1 year ago

First POC @ https://github.com/xp-framework/test

thekid commented 1 year ago

Migration script

thekid commented 1 year ago

Group (class) and case (method) prerequisites @ https://github.com/xp-framework/test/pull/1

thekid commented 1 year ago

Test prerequisites analysis

Usage of test actions:

address/.../RecordOfTest.class.php:73:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")'), Expect(class: Error::class, withMessage: '/Too few arguments .+/')]
address/.../RecordOfTest.class.php:78:  #[Test, Action(eval: 'new RuntimeVersion("<7.1")'), Expect(IllegalArgumentException::class)]
address/.../UsingNextTest.class.php:40:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
address/.../ValueOfTest.class.php:22:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
compiler/.../emit/EnumTest.class.php:8:#[Action(eval: 'new VerifyThat(fn() => function_exists("enum_exists"))')]
compiler/.../emit/IntersectionTypesTest.class.php:50:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/IntersectionTypesTest.class.php:62:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/IntersectionTypesTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
compiler/.../emit/InvocationTest.class.php:142:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
compiler/.../emit/InvocationTest.class.php:158:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
compiler/.../emit/LambdasTest.class.php:51:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(LambdaExpression::class, "static"))')]
compiler/.../emit/LambdasTest.class.php:62:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(ClosureExpression::class, "static"))')]
compiler/.../emit/ParameterTest.class.php:76:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
compiler/.../emit/ReflectionTest.class.php:43:  #[Test, Action(eval: 'new RuntimeVersion("<8.1")')]
compiler/.../emit/ReflectionTest.class.php:61:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
compiler/.../emit/TraitsTest.class.php:66:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:86:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:98:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../emit/UnionTypesTest.class.php:110:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
compiler/.../loader/CompilingClassLoaderTest.class.php:141:  #[Test, Action(eval: 'new RuntimeVersion(">=7.3")'), Expect(class: ClassFormatException::class, withMessage: '/Compiler error: Class .+ not found/')]
compression/.../CompressionTest.class.php:54:  #[Test, Values(['gzip', 'GZIP', '.gz', '.GZ']), Action(eval: 'new ExtensionAvailable("zlib")')]
core/.../annotations/BrokenAnnotationTest.class.php:31:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")'), Expect(['class' => ClassFormatException::class, 'withMessage' => '/Unterminated annotation/'])]
core/.../core/EnumTest.class.php:59:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:162:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:170:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:217:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:344:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/ErrorsTest.class.php:115:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new RuntimeVersion("<7.1.0-dev")')]
core/.../core/ErrorsTest.class.php:121:  #[Test, Expect(Error::class), Action(eval: 'new RuntimeVersion(">=7.1.0")')]
core/.../core/ErrorsTest.class.php:127:  #[Test, Expect(ClassCastException::class), Action(eval: 'new RuntimeVersion("<7.4.0-dev")')]
core/.../core/ErrorsTest.class.php:133:  #[Test, Expect(Error::class), Action(eval: 'new RuntimeVersion(">=7.4.0")')]
core/.../core/ExceptionsTest.class.php:140:  #[Test, Action(eval: 'new RuntimeVersion(">=7.0.0")')]
core/.../core/FunctionTypeTest.class.php:275:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../core/NewInstanceTest.class.php:202:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:224:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:246:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:258:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:270:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:282:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:390:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:405:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:420:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:437:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:453:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:468:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:483:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:498:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:513:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:528:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:543:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:559:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:574:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:589:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/ProcessResolveTest.class.php:41:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:46:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:51:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:58:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:65:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:72:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:77:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:87:  #[Test, Action(eval: 'new IsPlatform("WIN")'), Expect(IOException::class)]
core/.../core/ProcessResolveTest.class.php:107:  #[Test, Action(eval: 'new IsPlatform("ANDROID")')]
core/.../core/ProcessResolveTest.class.php:113:  #[Test, Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/ProcessResolveTest.class.php:118:  #[Test, Values(['"ls"', "'ls'"]), Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/ProcessResolveTest.class.php:123:  #[Test, Action(eval: 'new IsPlatform("WIN")')]
core/.../core/ProcessResolveTest.class.php:128:  #[Test, Action(eval: 'new IsPlatform("!(WIN|ANDROID)")')]
core/.../core/TypeIntersectionTest.class.php:120:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeIntersectionTest.class.php:127:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeIntersectionTest.class.php:134:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1.0-dev")')]
core/.../core/TypeOfTest.class.php:99:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../core/TypeOfTest.class.php:104:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../core/TypeOfTest.class.php:109:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeOfTest.class.php:114:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:192:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:201:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:210:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:219:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:228:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:234:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:240:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../core/TypeUnionTest.class.php:246:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0.0-dev")')]
core/.../io/PathTest.class.php:190:  #[Test, Action(eval: 'new IsPlatform("!^Win")')]
core/.../io/PathTest.class.php:256:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:266:  #[Test, Values(['C:', 'C:/', 'c:', 'C:/']), Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:347:  #[Test, Action(eval: 'new IsPlatform("^Win")'), Values([['\\\\remote\\file.txt', true], ['\\\\remote', true]])]
core/.../io/PathTest.class.php:352:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../io/PathTest.class.php:357:  #[Test, Action(eval: 'new IsPlatform("^Win")')]
core/.../reflection/FieldAccessTest.class.php:75:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldAccessTest.class.php:85:  #[Test, Expect(IllegalAccessException::class), Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldAccessTest.class.php:91:  #[Test, Expect(IllegalAccessException::class), Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldModifiersTest.class.php:28:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/FieldTypeTest.class.php:64:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/FieldTypeTest.class.php:77:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/FieldTypeTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/MethodInvocationTest.class.php:46:  #[Test, Expect(TargetInvocationException::class), Action(eval: 'new RuntimeVersion(">=7.0")')]
core/.../reflection/MethodInvocationTest.class.php:52:  #[Test, Expect(TargetInvocationException::class), Action(eval: 'new RuntimeVersion(">=7.0")')]
core/.../reflection/MethodParametersTest.class.php:97:  #[Test, Action(eval: 'new RuntimeVersion(">=7.0")'), Values([['string'], ['int'], ['bool'], ['float']])]
core/.../reflection/MethodParametersTest.class.php:105:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodParametersTest.class.php:114:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")'), Values([['string|int'], ['string|false']])]
core/.../reflection/MethodParametersTest.class.php:270:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:275:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:280:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:285:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodParametersTest.class.php:290:  #[Test, Action(eval: 'new RuntimeVersion("<8.0")')]
core/.../reflection/MethodReturnTypesTest.class.php:87:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:93:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:99:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/MethodReturnTypesTest.class.php:105:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")'), Values([['string|int'], ['string|false']])]
core/.../reflection/MethodReturnTypesTest.class.php:169:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/MethodReturnTypesTest.class.php:190:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/RuntimeClassDefinitionTest.class.php:148:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
core/.../reflection/TypeSyntaxTest.class.php:32:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
core/.../reflection/TypeSyntaxTest.class.php:53:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:60:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:67:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../reflection/TypeSyntaxTest.class.php:74:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
core/.../util/MoneyTest.class.php:8:#[Action(eval: 'new ExtensionAvailable("bcmath")')]
core/.../util/OpenSSLSecretTest.class.php:10:#[Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/PropertyExpansionTest.class.php:31:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../util/RandomTest.class.php:50:  #[Test, Action(eval: 'new VerifyThat(function() { return (function_exists("random_bytes") || function_exists("openssl_random_pseudo_bytes"));})')]
core/.../util/RandomTest.class.php:55:  #[Test, Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/RandomTest.class.php:60:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:93:  #[Test, Action(eval: 'new ExtensionAvailable("openssl")')]
core/.../util/RandomTest.class.php:99:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:116:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/RandomTest.class.php:121:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/SodiumSecretTest.class.php:10:#[Action(eval: 'new ExtensionAvailable("sodium")')]
coverage/.../coverage/tests/RecordCoverageTest.class.php:7:#[Action(eval: 'new VerifyThat(function() { return interface_exists(\unittest\Listener::class); })')]
ftp/.../IntegrationTest.class.php:15:#[Action(eval: 'new StartServer("peer.ftp.unittest.TestingServer", "connected", "shutdown")')]
imaging/.../ExifDataTest.class.php:14:#[Action(eval: 'new ExtensionAvailable("exif")')]
imaging/.../GifImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("GIF")]')]
imaging/.../GifImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("GIF")]')]
imaging/.../ImageReaderTest.class.php:16:#[Action(eval: 'new ExtensionAvailable("gd")')]
imaging/.../JpegImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("JPEG")]')]
imaging/.../JpegImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("JPEG")]')]
imaging/.../PngImageReaderTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("PNG")]')]
imaging/.../PngImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("PNG")]')]
imaging/.../WebpImageReaderTest.class.php:14:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("WEBP")]')]
imaging/.../WebpImageWriterTest.class.php:13:#[Action(eval: '[new ExtensionAvailable("gd"), new ImageTypeSupport("WEBP")]')]
mail/.../ImapStoreTest.class.php:13:#[Action(eval: 'new ExtensionAvailable("imap")')]
networking/.../sockets/BSDSocketTest.class.php:15:#[Action(eval: '[new ExtensionAvailable("sockets"), new StartServer(TestingServer::class, "connected", "shutdown")]')]
networking/.../sockets/SocketTest.class.php:12:#[Action(eval: 'new StartServer(TestingServer::class, "connected", "shutdown")')]
rdbms/.../ConfiguredConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../CriteriaTest.class.php:15:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../DataSetTest.class.php:16:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../DBTest.class.php:10:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../FinderTest.class.php:15:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../ProjectionTest.class.php:19:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../QueryTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../QueuedConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../RegisteredConnectionManagerTest.class.php:14:#[Action(eval: 'new RegisterMockConnection()')]
rdbms/.../sqlite3/SQLite3ConnectionTest.class.php:14:#[Action(eval: 'new ExtensionAvailable("sqlite3")')]
rdbms/.../sqlite3/SQLite3CreationTest.class.php:16:#[Action(eval: 'new ExtensionAvailable("sqlite3")')]
rdbms/.../StatementTest.class.php:13:#[Action(eval: 'new RegisterMockConnection()')]
reflection/.../ConstantsTest.class.php:30:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:56:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:62:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../ConstantsTest.class.php:68:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../InstantiationTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:89:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../MethodsTest.class.php:95:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:101:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:107:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:113:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")'), Values(['true', 'false'])]
reflection/.../MethodsTest.class.php:125:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1")')]
reflection/.../MethodsTest.class.php:228:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:237:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:246:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../MethodsTest.class.php:285:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../MethodsTest.class.php:294:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../ParametersTest.class.php:143:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:106:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")'), Expect(AccessingFailed::class)]
reflection/.../PropertiesTest.class.php:148:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:157:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:166:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:175:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:184:  #[Test, Action(eval: 'new RuntimeVersion(">=8.1")')]
reflection/.../PropertiesTest.class.php:193:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")'), Values(['true', 'false'])]
reflection/.../PropertiesTest.class.php:220:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../PropertiesTest.class.php:229:  #[Test, Action(eval: 'new RuntimeVersion(">=8.0")')]
reflection/.../PropertiesTest.class.php:238:  #[Test, Action(eval: 'new RuntimeVersion(">=7.4")')]
reflection/.../TypeTest.class.php:81:  #[Test, Action(eval: 'new RuntimeVersion(">=8.2")')]
reflection/.../TypeTest.class.php:111:  #[Test, Action(eval: 'new VerifyThat(fn() => !self::$ENUMS)')]
reflection/.../TypeTest.class.php:117:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:204:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:210:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
rest-client/.../CompressionHandlingTest.class.php:43:  #[Test, Action(eval: 'new ExtensionAvailable("zlib")')]
rest-client/.../CompressionHandlingTest.class.php:48:  #[Test, Action(eval: 'new ExtensionAvailable("brotli")')]
rest-client/.../CompressionHandlingTest.class.php:74:  #[Test, Values([1, 6, 9]), Action(eval: 'new ExtensionAvailable("zlib")')]
rest-client/.../CompressionHandlingTest.class.php:83:  #[Test, Values([0, 6, 11]), Action(eval: 'new ExtensionAvailable("brotli")')]
sequence/.../EachTest.class.php:85:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1.0")')]
sequence/.../PeekTest.class.php:76:  #[Test, Action(eval: 'new RuntimeVersion(">=7.1.0")')]
text-encode/.../Base64InputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
text-encode/.../Base64OutputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
webtest/.../web/tests/FormsTest.class.php:19:    $this->assertEquals($action, $form->getAction());
xml/.../XslCallbackTest.class.php:16:#[Action(eval: '[new ExtensionAvailable("dom"), new ExtensionAvailable("xsl")]')]
zip/.../AbstractZipFileTest.class.php:13:#[Action(eval: 'new ExtensionAvailable("zlib")')]
zip/.../vendors/SevenZipFileTest.class.php:36:  #[Test, Action(eval: 'new ExtensionAvailable("zlib")')]
zip/.../vendors/SevenZipFileTest.class.php:41:  #[Test, Action(eval: 'new ExtensionAvailable("bz2")')]

Usage of VerifyThat:

compiler/.../emit/EnumTest.class.php:8:#[Action(eval: 'new VerifyThat(fn() => function_exists("enum_exists"))')]
compiler/.../emit/LambdasTest.class.php:51:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(LambdaExpression::class, "static"))')]
compiler/.../emit/LambdasTest.class.php:62:  #[Test, Action(eval: 'new VerifyThat(fn() => property_exists(ClosureExpression::class, "static"))')]
core/.../core/EnumTest.class.php:59:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:162:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:170:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:217:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/EnumTest.class.php:344:  #[Test, Action(eval: 'new VerifyThat(fn() => class_exists("ReflectionEnum", false))')]
core/.../core/FunctionTypeTest.class.php:275:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../core/NewInstanceTest.class.php:202:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:224:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:246:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:258:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:270:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:282:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:390:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:405:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:420:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:437:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:453:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:468:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:483:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:498:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:513:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:528:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.2")]')]
core/.../core/NewInstanceTest.class.php:543:  #[Test, Action(eval: '[new VerifyThat("processExecutionEnabled"), new RuntimeVersion(">=7.1")]')]
core/.../core/NewInstanceTest.class.php:559:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:574:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../core/NewInstanceTest.class.php:589:  #[Test, Action(eval: 'new VerifyThat("processExecutionEnabled")')]
core/.../util/PropertyExpansionTest.class.php:31:  #[Test, Action(eval: 'new VerifyThat(fn() => !extension_loaded("xdebug"))')]
core/.../util/RandomTest.class.php:50:  #[Test, Action(eval: 'new VerifyThat(function() { return (function_exists("random_bytes") || function_exists("openssl_random_pseudo_bytes"));})')]
core/.../util/RandomTest.class.php:60:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:99:  #[Test, Action(eval: 'new VerifyThat(function() { return is_readable("/dev/urandom"); })')]
core/.../util/RandomTest.class.php:116:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
core/.../util/RandomTest.class.php:121:  #[Test, Expect(IllegalArgumentException::class), Action(eval: 'new VerifyThat(function() { return 0x7FFFFFFF === PHP_INT_MAX; })')]
coverage/.../RecordCoverageTest.class.php:7:#[Action(eval: 'new VerifyThat(function() { return interface_exists(\unittest\Listener::class); })')]
reflection/.../TypeTest.class.php:111:  #[Test, Action(eval: 'new VerifyThat(fn() => !self::$ENUMS)')]
reflection/.../TypeTest.class.php:117:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:204:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
reflection/.../TypeTest.class.php:210:  #[Test, Action(eval: 'new VerifyThat(fn() => self::$ENUMS)')]
text-encode/.../Base64InputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]
text-encode/.../Base64OutputStreamTest.class.php:13:#[Action(eval: 'new VerifyThat(fn() => in_array("convert.*", stream_get_filters()))')]

thekid commented 1 year ago

Transform values before comparison @ https://github.com/xp-framework/test/pull/3

thekid commented 1 year ago

https://github.com/xp-forge/assert has been set to public archive.

thekid commented 1 year ago

Submitted to Packagist -> https://packagist.org/packages/xp-framework/test

thekid commented 1 year ago

Generalized provider concept @ https://github.com/xp-framework/test/pull/4

thekid commented 1 year ago

First migration for Dialog done partially by migration script. The integration test needed to be migrated manually, as constructor arguments are not passed by default - see https://github.com/xp-framework/test/pull/7

thekid commented 1 year ago

Current library is less than 1000 LOC, compared to 2974 for xp-framework/unittest:

$ cloc-1.82.pl src/main/php/
      39 text files.
      39 unique files.
       9 files ignored.

github.com/AlDanial/cloc v 1.82  T=0.10 s (404.0 files/s, 15444.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
PHP                             39            221            368            902
-------------------------------------------------------------------------------
SUM:                            39            221            368            902
-------------------------------------------------------------------------------

thekid commented 1 year ago

First release @ https://github.com/xp-framework/test/releases/tag/v1.0.0 🎉

thekid commented 1 year ago

Early adapters:

thekid commented 1 year ago

Project created for tracking progress: https://github.com/orgs/xp-framework/projects/3/views/1?layout=board

thekid commented 1 year ago

All of xp-lang is now migrated, see here

thekid commented 1 year ago

All of xp-forge is now migrated, see here

thekid commented 1 year ago

Finally, XP Core is migrated 🎉

thekid commented 1 year ago

The repository https://github.com/xp-framework/unittest is now archived

thekid commented 4 months ago

Remaining libraries:

$ grep 'xp-framework/unittest' */composer.json | grep -v ^unittest
caching/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
coverage/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.7",
ftp/composer.json:    "xp-framework/unittest": "^11.0 | ^10.1"
imaging/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
io-collections/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0 | ^6.5"
ldap/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
mail/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
measure/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
meilisearch/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
mocks/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 |^7.0",
patterns/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0 | ^7.0"
sql-parser/composer.json:    "xp-framework/unittest": "^11.1"
text-encode/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0"
webtest/composer.json:    "xp-framework/unittest": "^11.0 | ^10.0 | ^9.0 | ^8.0",

xp-framework / rfc

New testing library #344

Scope of Change

Rationale

Functionality

Naming

Assert DSL

Extended assertions

Fluent assertions

Transformation

Values and providers

Provider implementation example

Test prerequisites

Security considerations

Speed impact

Dependencies

Related documents

Migration script

Test prerequisites analysis

First release @ https://github.com/xp-framework/test/releases/tag/v1.0.0 🎉

Finally, XP Core is migrated 🎉