tripu commented 5 years ago

As I wrote on the document, imho testharness.js needs at the very least to be a project maintained independently, and a package available via npm. Then, we will need good reasons to abandon the very popular mocha in favour of this other (less powerful?) tool.

ping @plehegar

ericprud commented 5 years ago

Jest

AFAIKT, web-platform testing isn't aiming for general use while Jest is. Here's my experience with using Jest on a couple projects:

⊖ ~2s slower to start than mocha so had to increase timeouts
⊕ despite warm-up time, almost twice as fast on large-ish test suite (tests, harness)
⊕ embeds coverage (Istanbul)
⊘ describe/test structure similar to mocha:

mocha	jest
`describe(desc, () => { this.timeout(T1); ... })`	`describe(desc, () => { ... }, T1)`
`it(desc, () => {}).timeout(T2)`	`it(desc, () => {}, T2)` (or `test(...)`)
`before(done => {...done();})`	`beforeAll(done => {...done();}, T2)`
`beforeEach`	`beforeEach`

⊕ assertion framework requires less ceremony than chai:

chai expect	jest expect
`expect(res).to.match(ref)`	`expect(res).toMatch(ref)`
`expect(res).to.equal(ref)`	`expect(res).toBe(ref)`
`expect(res).to.deep.equal(ref)`	`expect(res).toEqual(ref)`
`res.should.exist`	`expect(res).toBeDefined()`
`res.to.be.undefined`	`expect(res).toBeUndefined()`
`res.should.be.a('object')`	`expect(res).toBe('object')`
`expect(f).to.throw(Error, /pattern/)`	`expect(f).toThrowError(Error)` `toThrowError` takes one argument (see facebook/jest#7000)
...	...

I've not used Jest in the browser (supposed to be good with Angular).

Hopefully, this will save some folks some aggravation.

tripu commented 5 years ago

Thank you, @ericprud, for contributing — and for doing so with concrete ideas and details!

I haven't used Jest, but I know it's an alternative that's popular with the community. So I'm not opposed in principle. Let's see what others have to say…

Broader question: how to decide when it's time we recommend (or at least we “accept”) a new package? I'd say we can consider alternatives or additions when there is at least one W3C project using it, and its maintainers have that experience and can defend its usage with specific arguments.

I see there is not one public W3C project using Jest yet. I'll be happy for a dissenter to break with the orthodoxy and adopt it themselves on a public W3C software project, then report back here on advantages (as @ericprud did).

SimenB commented 5 years ago

I've not used Jest in the browser (supposed to be good with Angular).

Note that Jest cannot run in a real browser (yet: https://github.com/facebook/jest/issues/848), it runs "browser tests" in JSDOM.

You can use its assertions standalone though, instead of e.g. chai in mocha: https://www.npmjs.com/package/expect. Those support the browser

ericprud commented 5 years ago

In jsdom with nodes earlier than 8(?), you can't let element names collide with function names, c.f. https://github.com/jsdom/jsdom/issues/2259 .

What's the typical workflow for such in-browser tests, does it use headless-browser? (I've only managed to automate browser tests using jsdom so far.)

w3c / nodejs

Testing: recommend “testharness.js” instead of “mocha”? #4

Jest