Opening up a Public API

[nicojs]

It would be great if we could use WCT programmatically using a public API. I was looking to the readme, but i didn't find any documentation on this. Does an undocumented api exist? If not, are you open for a pull request?

The way i see it, it can be fairly straight forward to open up the steps used in tests and expose them as the public api.

Background info on our use case:

We want to support WCT as one of the test runners in Stryker, the JavaScript mutation testing framework. In order to do that, it would be helpful if we could use an api to open the test runner (browser) and run the tests multiple times while writing different mutations of the source code to disk. After each test run, we would like to know which tests ran and if they passed or failed.

[nicojs]

Any updates on this point? I want to do the work, but can anyone point me in the right direction? Thanks in advance.

[justinfagnani]

Hi Nico. I think we'd be open to a more detailed design/proposal and PRs. We can use this issue to discuss what the API would look like, or a new one if you wish.

[nicojs]

@justinfagnani thanks for your response. Let's start the discussion here, sounds great.

It could look something like this:

import { Config } from './config';

type RunStatus = 'complete' | 'error' | 'timeout';

interface RunResult {
    // The status of the test run
    status: RunStatus;
    // The individual test results
    tests: TestResult[];
    // Optional error messages when status = 'Error'
    errorMessages?: string[];
}

type TestStatus = 'success' | 'failed' | 'skipped';

interface TestResult {
    // The full human readable name of the test
    name: string;
    // The status of the test
    status: TestStatus;
    // The time it took to run the test
    timeSpentMs: number;
    // Optional: messages in case of status: Failed
    failureMessages?: string[];
}

interface WebComponentTester {

    new (options: Config): WebComponentTester;

    // Starts up all browsers and resolves when done
    initialize(): Promise<void>;

    // Starts a test run and reports the result when done
    run(): Promise<RunResult>;

    // Closes down all browsers and resolves when done
    close(): Promise<void>;
}

I've created this draft with our own TestRunner interface in mind. We work with Promises instead of events, however i see the web component tester is event driven, so maybe you would want to see a more event based approach?

As long as we can make the translation between our TestRunner interface and this api i will be happy. Further more, as we do hundreds of test runs for average projects, we want to reduce browser overhead as much as possible. The idea would be that we start a browser once and do multiple runs in quick succession while changing the content of 1 or 2 source files between runs.

[nicojs]

@justinfagnani Any updates on this?

[justinfagnani]

@nicojs sorry for the delay. This API looks reasonable. I like the Promise-based API. Events are fine too for repeated things, and I could see wanting a streaming API for test results. If we considered that for a v2 of the API, how would it fit on top of this?

[nicojs]

@justinfagnani sorry for the delay on my and :). We've been very busy. I hope i can fit wct support into our backlog in the comming 2 months (it's a part time/spare time effort).

If v2 would introduce a streaming api, that would be great. For Stryker it would mean updating the wct adapter we would have in place by than with the new streaming stuff. No big deal.

Did anything change in the mean time? Or is the API proposal still up-to-date?

[justinfagnani]

@nicojs no problem, I know how this goes :)

Everything still looks ok here. The only development is that longer term we're thinking (only vaguely at this point) of restructuring WCT with probably some of the same goals you have in mind here. I wonder if we should chat about that.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.