search

haxe-utest / utest

cross-platform unit testing system for Haxe
96 stars 36 forks source link

API to expose test results #75

Open RealyUniqueName opened 5 years ago

RealyUniqueName commented 5 years ago

Currently, there is no way to access test results. Something like this is needed:

public var success(default,null):Bool;
public getFailures():FailuresData;

maybe as an addition to utest.ui.common.IReport

Requested here:

#if js
    static function nodejsMain() {
        main();
        (untyped process).exit(Test.success ? 0 : 1);
    }
    #end

I couldn't immediately find how to get the success state of the tests. (quote from https://github.com/HaxeFoundation/haxe/pull/7901#issue-257273666)

Simn commented 5 years ago

I realized that PlainTextReport has a built-in Sys.exit() in its complete function. So I'm not sure if this is a valid issue because the API clearly wants me to handle this differently.

RealyUniqueName commented 5 years ago

Well, yes. You are expected to implement your own IReport. But that's not very convenient when you just want to handle success/fail state or amount of failures.

Simn commented 5 years ago

My point is that you can't handle success/fail yourself anyway if the thing Sys.exits on you.

RealyUniqueName commented 5 years ago

utest.Runner has callback onComplete which is called before the shutdown.

Simn commented 5 years ago

Looks like we indeed need this for https://github.com/HaxeFoundation/haxe/commit/6924babc1ceed3b3821d53669b1ac8474c8e370f