Propose testing tools and frameworks

[KatieUde]

General Considerations for a Test Framework:

1. It should be fast
2. It should easily handle asynchronous tests
3. It should be easy to use from the command line
4. It should allow one to use any assertion and mock library
5. It should work in the browser for ease of debugging

The following provides some options based upon research. However, I think we ultimately will want to decide if we pick tools to use universally or if we tailor to the specific project at hand.

Option 1:

• Mocha: JavaScript test framework; one of the two most popular testing frameworks
• Chai: Assertion/expectation library that can be paired with any JavaScript testing framework
• Sinon: Test spies, stubs and mocks for JavaScript; no dependencies, works with any unit testing framework
• Karma: Tool that allows for fast test reloads, multiple browser testing, and webpack preprocessing
• Enzyme: JavaScript Testing utility for React that makes it easier to assert, manipulate, and traverse your React Components' output (obviously only needed if a React project)

Option 2:

• Tape: Unit testing library that provides out of the box a test runner, an assertion library, and formatted results using the standard TAP(Test Anything Protocol)
• Tape-jsx-equals: Tape extension to compare React components
• Karma: Tool that allows for fast test reloads, multiple browser testing, and webpack preprocessing

Option 3:

• Jest: JavaScript test framework; used by Facebook to test all JavaScript code including React applications
• Enzyme: JavaScript Testing utility for React that makes it easier to assert, manipulate, and traverse your React Components' output (obviously only needed if a React project)
• Karma: Tool that allows for fast test reloads, multiple browser testing, and webpack preprocessing

These three options are not meant to be a comprehensive listing of offerings available for use. There are many more that could be considered, but based on frameworks we typically use (angular and/or react), the ones suggested seemed to make the most sense. If we want to pick a tool to use universally, I am leaning towards the Mocha/Chai route, but there are pros and cons to each of the options I provided.

[KatieUde]

Mocha

Pros:

• Very popular in js community
• More configurable than Jasmine
• Flexible (can be used w/ any assertion, mock, report library)
• Clear, simple API
• Asynchronous code testing support
• Fast test run

Cons:

• No automocking or snapshot testing
• Doesn't have built-in assertion library
• Thought by some to have too much configuration/options (leads to "analysis paralysis")
• Have a lot of global functions that "litter" the global namespace; contradictory to best practices that encourage avoiding use of globals

Tape

Pros:

• Simplest & most flexible test library for JavaScript that runs on Node & browsers
• Allows for more modular testing; does not add the global namespace or global functions
• Built-in assert (don't have to add more dependencies and external assertion libraries)
• Follows the principles of TAP (relevant if used before)

  Cons:

• No concurrency; cannot run async test cases concurrently
• Needs to know when all the assertions for each test case are done (ex: call the end() method on the execution object in your test callback

Jest

Pros:

• Officially supported by React dev/Facebook
• Snapshot testing of React trees or other serializable values
• Asynchronous code testing support

  Cons:

• Slower if using auto mocking
• Based on Jasmine
• Not great documentation
• Auto mocking

[jschroed91]

@KatieUde you rock!!

I am also leaning towards the mocha route (option 1).

It plays nicely with webpack I assume based on the webpack preprocess.

What is auto mocking in Jest? In curious what it is and why it's a con - auto mocking sounds like a lazy developers best friend!! ;) But seriously, I haven't heard it before so I am curious.

Waiting for hear back on Beck and Chris' feedback too - and we can determine next steps.

I'll have a PR for a basic webpack set up tomorrow... that should at least unblock us from implementing some test examples - that would be the next step - and if we're unable to pin down on one right now, then maybe we do example implementations of the same test in 2 of the 3 options (Jest is appealing to me but it sounds like it might be tough to work with off the bat, and might be biased toward React apps.

[jschroed91]

Some quick research on auto mocking has made it apparent why it's listed as a con 👅

My initial vote is for Mocha based on this! I await the feedback of my peers

[jschroed91]

Based on discussion in the hipchat room and elsewhere, I think we're going to go with option 1 for our first go and we'll see how it goes.