Recommending an automated test framework and config

[ericelliott]

With all the different options and permutations, setting up an automated testing framework can be one of the most daunting tasks for a team building a new application, or upgrading the process for legacy apps.

In some projects, I've seen a mix of BDD and TDD styles for unit tests, and a barely cohesive and brittle jumble of functional and integration tests. It would be great to achieve some consensus on best approaches to tame this jungle.

I want to make it a lot more clear how people should organize their tests, why it's important to draw a clear line between unit tests and functional / integration tests, and call out testing issues that come up for most projects.

I'd love to hear your thoughts, particularly answers to these questions:

• What are your biggest pain points around test suites?
• Do you use TDD or BDD, or both? Why?
• Do you use a unified, cohesive framework for both your unit tests and functional tests?
  • Are there advantages to that over treating them as separate concerns?
  • What are those advantages?
  • What are the disadvantages?
• I very recently learned there may be some advantages to writing tests in AMD format instead of node-style modules or ES6. What is your experience, particularly around conditional require syntax?
  • Does AMD-style testing significantly improve isomorphic testing for your projects?
  • Are there drawbacks to using AMD as the test authoring format? If so, what are they?

[neophob]

mockery to mock out require calls (and disable the require cache), sinon to mock/spy/stub, chai expect and mocha.

[ericelliott]

Hi @neophob,

Thanks for your feedback!

What are your favorite things about that setup?

What would you change if you could?

• Eric

[ericelliott]

I have been using tape for years and it has never let me down.

Here's what I like about it:

• Produces simple tap output that is compatible with a wide variety of tools
• Easy to set up and integrate with CI tools
• Dead simple assertions with no ambiguity or magic

Here's what I think is missing -- super obvious before / after / beforeEach / afterEach. Easily corrected:

// credit: https://github.com/substack/tape/issues/59
var test = require("tape")

var before = test
var after = test

var test = beforeEach(test, function before(assert) {
    // called before each thing

    // when done call
    assert.end()
})
test = afterEach(test, function after(assert) {
    // called after each thing

    // when done call
    assert.end()
})

before("before all tests", function (assert) {
    // before logic

    // when done call
    assert.end()
})

test("name of test", function (assert) {
    // before logic & beforeEach has run

    // when done with test call
    assert.end()
    // afterEach logic will run
})

after("after all tests", function (assert) {
    // after logic

    // when done call
    assert.end()
})

function beforeEach(test, handler) {
    return function tapish(name, listener) {
        test(name, function (assert) {
            var _end = assert.end
            assert.end = function () {
                assert.end = _end
                listener(assert)
            }

            handler(assert)
        })
    }
}
function afterEach(test, handler) {
    return function tapish(name, listener) {
        test(name, function (assert) {
            var _end = assert.end
            assert.end = function () {
                assert.end = _end
                handler(assert)
            }

            listener(assert)
        })
    }
}

If you're interested in checking out the tape workflow, you'll also want to look at faucet and browserify.

[ivanoats]

For me Mocha/Chai/Sinon are the tools that have stood the test of time in the JavaScript community. I don't have specific stats but it's probably 49% Jasmine 51% Mocha/Chai/Sinon that I see in various projects. My set up for Server/Unit/Acceptance tests is here: https://github.com/UWFosterIT/react-starter/tree/master/test