nodejs / node-core-test

Node 18's node:test, as an npm package
Other
95 stars 10 forks source link
nodejs testing

The test npm package

CI

This is a user-land port of node:test, the experimental test runner introduced in Node.js 18. This module makes it available in Node.js 14 and later.

Minimal dependencies, with full test suite.

Differences from the core implementation:

Docs

Test runner

Stability: 1 - Experimental

The node:test module facilitates the creation of JavaScript tests. To access it:

import test from 'test'
const test = require('test')

Tests created via the test module consist of a single function that is processed in one of three ways:

  1. A synchronous function that is considered failing if it throws an exception, and is considered passing otherwise.
  2. A function that returns a Promise that is considered failing if the Promise rejects, and is considered passing if the Promise resolves.
  3. A function that receives a callback function. If the callback receives any truthy value as its first argument, the test is considered failing. If a falsy value is passed as the first argument to the callback, the test is considered passing. If the test function receives a callback function and also returns a Promise, the test will fail.

The following example illustrates how tests are written using the test module.

test('synchronous passing test', t => {
  // This test passes because it does not throw an exception.
  assert.strictEqual(1, 1)
})

test('synchronous failing test', t => {
  // This test fails because it throws an exception.
  assert.strictEqual(1, 2)
})

test('asynchronous passing test', async t => {
  // This test passes because the Promise returned by the async
  // function is not rejected.
  assert.strictEqual(1, 1)
})

test('asynchronous failing test', async t => {
  // This test fails because the Promise returned by the async
  // function is rejected.
  assert.strictEqual(1, 2)
})

test('failing test using Promises', t => {
  // Promises can be used directly as well.
  return new Promise((resolve, reject) => {
    setImmediate(() => {
      reject(new Error('this will cause the test to fail'))
    })
  })
})

test('callback passing test', (t, done) => {
  // done() is the callback function. When the setImmediate() runs, it invokes
  // done() with no arguments.
  setImmediate(done)
})

test('callback failing test', (t, done) => {
  // When the setImmediate() runs, done() is invoked with an Error object and
  // the test fails.
  setImmediate(() => {
    done(new Error('callback failure'))
  })
})

If any tests fail, the process exit code is set to 1.

Subtests

The test context's test() method allows subtests to be created. This method behaves identically to the top level test() function. The following example demonstrates the creation of a top level test with two subtests.

test('top level test', async t => {
  await t.test('subtest 1', t => {
    assert.strictEqual(1, 1)
  })

  await t.test('subtest 2', t => {
    assert.strictEqual(2, 2)
  })
})

In this example, await is used to ensure that both subtests have completed. This is necessary because parent tests do not wait for their subtests to complete. Any subtests that are still outstanding when their parent finishes are cancelled and treated as failures. Any subtest failures cause the parent test to fail.

Skipping tests

Individual tests can be skipped by passing the skip option to the test, or by calling the test context's skip() method as shown in the following example.

// The skip option is used, but no message is provided.
test('skip option', { skip: true }, t => {
  // This code is never executed.
})

// The skip option is used, and a message is provided.
test('skip option with message', { skip: 'this is skipped' }, t => {
  // This code is never executed.
})

test('skip() method', t => {
  // Make sure to return here as well if the test contains additional logic.
  t.skip()
})

test('skip() method with message', t => {
  // Make sure to return here as well if the test contains additional logic.
  t.skip('this is skipped')
})

describe/it syntax

Running tests can also be done using describe to declare a suite and it to declare a test. A suite is used to organize and group related tests together. it is an alias for test, except there is no test context passed, since nesting is done using suites.

describe('A thing', () => {
  it('should work', () => {
    assert.strictEqual(1, 1);
  });

  it('should be ok', () => {
    assert.strictEqual(2, 2);
  });

  describe('a nested thing', () => {
    it('should work', () => {
      assert.strictEqual(3, 3);
    });
  });
});

describe and it are imported from the test module.

import { describe, it } from 'test';
const { describe, it } = require('test');

only tests

If node--test is started with the --test-only command-line option, it is possible to skip all top level tests except for a selected subset by passing the only option to the tests that should be run. When a test with the only option set is run, all subtests are also run. The test context's runOnly() method can be used to implement the same behavior at the subtest level.

// Assume node--test is run with the --test-only command-line option.
// The 'only' option is set, so this test is run.
test('this test is run', { only: true }, async t => {
  // Within this test, all subtests are run by default.
  await t.test('running subtest')

  // The test context can be updated to run subtests with the 'only' option.
  t.runOnly(true)
  await t.test('this subtest is now skipped')
  await t.test('this subtest is run', { only: true })

  // Switch the context back to execute all tests.
  t.runOnly(false)
  await t.test('this subtest is now run')

  // Explicitly do not run these tests.
  await t.test('skipped subtest 3', { only: false })
  await t.test('skipped subtest 4', { skip: true })
})

// The 'only' option is not set, so this test is skipped.
test('this test is not run', () => {
  // This code is not run.
  throw new Error('fail')
})

Filtering tests by name

The [--test-name-pattern][] command-line option can be used to only run tests whose name matches the provided pattern. Test name patterns are interpreted as JavaScript regular expressions. The --test-name-pattern option can be specified multiple times in order to run nested tests. For each test that is executed, any corresponding test hooks, such as beforeEach(), are also run.

Given the following test file, starting Node.js with the --test-name-pattern="test [1-3]" option would cause the test runner to execute test 1, test 2, and test 3. If test 1 did not match the test name pattern, then its subtests would not execute, despite matching the pattern. The same set of tests could also be executed by passing --test-name-pattern multiple times (e.g. --test-name-pattern="test 1", --test-name-pattern="test 2", etc.).

test('test 1', async (t) => {
  await t.test('test 2');
  await t.test('test 3');
});
test('Test 4', async (t) => {
  await t.test('Test 5');
  await t.test('test 6');
});

Test name patterns can also be specified using regular expression literals. This allows regular expression flags to be used. In the previous example, starting Node.js with --test-name-pattern="/test [4-5]/i" would match Test 4 and Test 5 because the pattern is case-insensitive.

Test name patterns do not change the set of files that the test runner executes.

Extraneous asynchronous activity

Once a test function finishes executing, the results are reported as quickly as possible while maintaining the order of the tests. However, it is possible for the test function to generate asynchronous activity that outlives the test itself. The test runner handles this type of activity, but does not delay the reporting of test results in order to accommodate it.

In the following example, a test completes with two setImmediate() operations still outstanding. The first setImmediate() attempts to create a new subtest. Because the parent test has already finished and output its results, the new subtest is immediately marked as failed, and reported later to the {TestsStream}.

The second setImmediate() creates an uncaughtException event. uncaughtException and unhandledRejection events originating from a completed test are marked as failed by the test module and reported as diagnostic warnings at the top level by the {TestsStream}.

test('a test that creates asynchronous activity', t => {
  setImmediate(() => {
    t.test('subtest that is created too late', t => {
      throw new Error('error1')
    })
  })

  setImmediate(() => {
    throw new Error('error2')
  })

  // The test finishes after this line.
})

Running tests from the command line

The Node.js test runner can be invoked from the command line:

node--test

By default, Node.js will recursively search the current directory for JavaScript source files matching a specific naming convention. Matching files are executed as test files. More information on the expected test file naming convention and behavior can be found in the test runner execution model section.

Alternatively, one or more paths can be provided as the final argument(s) to the Node.js command, as shown below.

node--test test1.js test2.mjs custom_test_dir/
node--test test1.js test2.mjs custom_test_dir/

In this example, the test runner will execute the files test1.js and test2.mjs. The test runner will also recursively search the custom_test_dir/ directory for test files to execute.

Test runner execution model

When searching for test files to execute, the test runner behaves as follows:

Each matching test file is executed in a separate child process. If the child process finishes with an exit code of 0, the test is considered passing. Otherwise, the test is considered to be a failure. Test files must be executable by Node.js, but are not required to use the node:test module internally.

Mocking

The node:test module supports mocking during testing via a top-level mock object. The following example creates a spy on a function that adds two numbers together. The spy is then used to assert that the function was called as expected.

import assert from 'node:assert';
import { mock, test } from 'test';
test('spies on a function', () => {
  const sum = mock.fn((a, b) => {
    return a + b;
  });
  assert.strictEqual(sum.mock.calls.length, 0);
  assert.strictEqual(sum(3, 4), 7);
  assert.strictEqual(sum.mock.calls.length, 1);
  const call = sum.mock.calls[0];
  assert.deepStrictEqual(call.arguments, [3, 4]);
  assert.strictEqual(call.result, 7);
  assert.strictEqual(call.error, undefined);
  // Reset the globally tracked mocks.
  mock.reset();
});
'use strict';
const assert = require('node:assert');
const { mock, test } = require('test');
test('spies on a function', () => {
  const sum = mock.fn((a, b) => {
    return a + b;
  });
  assert.strictEqual(sum.mock.calls.length, 0);
  assert.strictEqual(sum(3, 4), 7);
  assert.strictEqual(sum.mock.calls.length, 1);
  const call = sum.mock.calls[0];
  assert.deepStrictEqual(call.arguments, [3, 4]);
  assert.strictEqual(call.result, 7);
  assert.strictEqual(call.error, undefined);
  // Reset the globally tracked mocks.
  mock.reset();
});

The same mocking functionality is also exposed on the TestContext object of each test. The following example creates a spy on an object method using the API exposed on the TestContext. The benefit of mocking via the test context is that the test runner will automatically restore all mocked functionality once the test finishes.

test('spies on an object method', (t) => {
  const number = {
    value: 5,
    add(a) {
      return this.value + a;
    },
  };
  t.mock.method(number, 'add');
  assert.strictEqual(number.add.mock.calls.length, 0);
  assert.strictEqual(number.add(3), 8);
  assert.strictEqual(number.add.mock.calls.length, 1);
  const call = number.add.mock.calls[0];
  assert.deepStrictEqual(call.arguments, [3]);
  assert.strictEqual(call.result, 8);
  assert.strictEqual(call.target, undefined);
  assert.strictEqual(call.this, number);
});

Test reporters

The node:test module supports passing [--test-reporter][] flags for the test runner to use a specific reporter.

The following built-reporters are supported:

Custom reporters

[--test-reporter][] can be used to specify a path to custom reporter. a custom reporter is a module that exports a value accepted by [stream.compose][]. Reporters should transform events emitted by a {TestsStream}

Example of a custom reporter using {stream.Transform}:

import { Transform } from 'node:stream';
const customReporter = new Transform({
  writableObjectMode: true,
  transform(event, encoding, callback) {
    switch (event.type) {
      case 'test:start':
        callback(null, `test ${event.data.name} started`);
        break;
      case 'test:pass':
        callback(null, `test ${event.data.name} passed`);
        break;
      case 'test:fail':
        callback(null, `test ${event.data.name} failed`);
        break;
      case 'test:plan':
        callback(null, 'test plan');
        break;
      case 'test:diagnostic':
        callback(null, event.data.message);
        break;
    }
  },
});
export default customReporter;
const { Transform } = require('node:stream');
const customReporter = new Transform({
  writableObjectMode: true,
  transform(event, encoding, callback) {
    switch (event.type) {
      case 'test:start':
        callback(null, `test ${event.data.name} started`);
        break;
      case 'test:pass':
        callback(null, `test ${event.data.name} passed`);
        break;
      case 'test:fail':
        callback(null, `test ${event.data.name} failed`);
        break;
      case 'test:plan':
        callback(null, 'test plan');
        break;
      case 'test:diagnostic':
        callback(null, event.data.message);
        break;
    }
  },
});
module.exports = customReporter;

Example of a custom reporter using a generator function:

export default async function * customReporter(source) {
  for await (const event of source) {
    switch (event.type) {
      case 'test:start':
        yield `test ${event.data.name} started\n`;
        break;
      case 'test:pass':
        yield `test ${event.data.name} passed\n`;
        break;
      case 'test:fail':
        yield `test ${event.data.name} failed\n`;
        break;
      case 'test:plan':
        yield 'test plan';
        break;
      case 'test:diagnostic':
        yield `${event.data.message}\n`;
        break;
    }
  }
}
module.exports = async function * customReporter(source) {
  for await (const event of source) {
    switch (event.type) {
      case 'test:start':
        yield `test ${event.data.name} started\n`;
        break;
      case 'test:pass':
        yield `test ${event.data.name} passed\n`;
        break;
      case 'test:fail':
        yield `test ${event.data.name} failed\n`;
        break;
      case 'test:plan':
        yield 'test plan\n';
        break;
      case 'test:diagnostic':
        yield `${event.data.message}\n`;
        break;
    }
  }
};

The value provided to --test-reporter should be a string like one used in an import() in JavaScript code, or a value provided for [--import][].

Multiple reporters

The [--test-reporter][] flag can be specified multiple times to report test results in several formats. In this situation it is required to specify a destination for each reporter using [--test-reporter-destination][]. Destination can be stdout, stderr, or a file path. Reporters and destinations are paired according to the order they were specified.

In the following example, the spec reporter will output to stdout, and the dot reporter will output to file.txt:

node --test-reporter=spec --test-reporter=dot --test-reporter-destination=stdout --test-reporter-destination=file.txt

When a single reporter is specified, the destination will default to stdout, unless a destination is explicitly provided.

run([options])

run({ files: [path.resolve('./tests/test.js')] })
  .pipe(process.stdout);

test([name][, options][, fn])

The test() function is the value imported from the test module. Each invocation of this function results in reporting the test to the {TestsStream}.

The TestContext object passed to the fn argument can be used to perform actions related to the current test. Examples include skipping the test, adding additional diagnostic information, or creating subtests.

test() returns a Promise that resolves once the test completes. The return value can usually be discarded for top level tests. However, the return value from subtests should be used to prevent the parent test from finishing first and cancelling the subtest as shown in the following example.

test('top level test', async t => {
  // The setTimeout() in the following subtest would cause it to outlive its
  // parent test if 'await' is removed on the next line. Once the parent test
  // completes, it will cancel any outstanding subtests.
  await t.test('longer running subtest', async t => {
    return new Promise((resolve, reject) => {
      setTimeout(resolve, 1000)
    })
  })
})

The timeout option can be used to fail the test if it takes longer than timeout milliseconds to complete. However, it is not a reliable mechanism for canceling tests because a running test might block the application thread and thus prevent the scheduled cancellation.

describe([name][, options][, fn])

The describe() function imported from the test module. Each invocation of this function results in the creation of a Subtest. After invocation of top level describe functions, all top level tests and suites will execute.

describe.skip([name][, options][, fn])

Shorthand for skipping a suite, same as describe([name], { skip: true }[, fn]).

describe.todo([name][, options][, fn])

Shorthand for marking a suite as TODO, same as describe([name], { todo: true }[, fn]).

it([name][, options][, fn])

The it() function is the value imported from the test module.

it.skip([name][, options][, fn])

Shorthand for skipping a test, same as it([name], { skip: true }[, fn]).

it.todo([name][, options][, fn])

Shorthand for marking a test as TODO, same as it([name], { todo: true }[, fn]).

before([, fn][, options])

This function is used to create a hook running before running a suite.

describe('tests', async () => {
  before(() => console.log('about to run some test'));
  it('is a subtest', () => {
    assert.ok('some relevant assertion here');
  });
});

after([, fn][, options])

This function is used to create a hook running after running a suite.

describe('tests', async () => {
  after(() => console.log('finished running tests'));
  it('is a subtest', () => {
    assert.ok('some relevant assertion here');
  });
});

beforeEach([, fn][, options])

This function is used to create a hook running before each subtest of the current suite.

describe('tests', async () => {
  beforeEach(() => t.diagnostics('about to run a test'));
  it('is a subtest', () => {
    assert.ok('some relevant assertion here');
  });
});

afterEach([, fn][, options])

This function is used to create a hook running after each subtest of the current test.

describe('tests', async () => {
  afterEach(() => t.diagnostics('about to run a test'));
  it('is a subtest', () => {
    assert.ok('some relevant assertion here');
  });
});

Class: MockFunctionContext

The MockFunctionContext class is used to inspect or manipulate the behavior of mocks created via the MockTracker APIs.

ctx.calls

A getter that returns a copy of the internal array used to track calls to the mock. Each entry in the array is an object with the following properties.

ctx.callCount()

This function returns the number of times that this mock has been invoked. This function is more efficient than checking ctx.calls.length because ctx.calls is a getter that creates a copy of the internal call tracking array.

ctx.mockImplementation(implementation)

This function is used to change the behavior of an existing mock.

The following example creates a mock function using t.mock.fn(), calls the mock function, and then changes the mock implementation to a different function.

test('changes a mock behavior', (t) => {
  let cnt = 0;
  function addOne() {
    cnt++;
    return cnt;
  }
  function addTwo() {
    cnt += 2;
    return cnt;
  }
  const fn = t.mock.fn(addOne);
  assert.strictEqual(fn(), 1);
  fn.mock.mockImplementation(addTwo);
  assert.strictEqual(fn(), 3);
  assert.strictEqual(fn(), 5);
});

ctx.mockImplementationOnce(implementation[, onCall])

This function is used to change the behavior of an existing mock for a single invocation. Once invocation onCall has occurred, the mock will revert to whatever behavior it would have used had mockImplementationOnce() not been called.

The following example creates a mock function using t.mock.fn(), calls the mock function, changes the mock implementation to a different function for the next invocation, and then resumes its previous behavior.

test('changes a mock behavior once', (t) => {
  let cnt = 0;
  function addOne() {
    cnt++;
    return cnt;
  }
  function addTwo() {
    cnt += 2;
    return cnt;
  }
  const fn = t.mock.fn(addOne);
  assert.strictEqual(fn(), 1);
  fn.mock.mockImplementationOnce(addTwo);
  assert.strictEqual(fn(), 3);
  assert.strictEqual(fn(), 4);
});

ctx.restore()

Resets the implementation of the mock function to its original behavior. The mock can still be used after calling this function.

Class: MockTracker

The MockTracker class is used to manage mocking functionality. The test runner module provides a top level mock export which is a MockTracker instance. Each test also provides its own MockTracker instance via the test context's mock property.

mock.fn([original[, implementation]][, options])

This function is used to create a mock function.

The following example creates a mock function that increments a counter by one on each invocation. The times option is used to modify the mock behavior such that the first two invocations add two to the counter instead of one.

test('mocks a counting function', (t) => {
  let cnt = 0;
  function addOne() {
    cnt++;
    return cnt;
  }
  function addTwo() {
    cnt += 2;
    return cnt;
  }
  const fn = t.mock.fn(addOne, addTwo, { times: 2 });
  assert.strictEqual(fn(), 2);
  assert.strictEqual(fn(), 4);
  assert.strictEqual(fn(), 5);
  assert.strictEqual(fn(), 6);
});

mock.getter(object, methodName[, implementation][, options])

This function is syntax sugar for MockTracker.method with options.getter set to true.

mock.method(object, methodName[, implementation][, options])

This function is used to create a mock on an existing object method. The following example demonstrates how a mock is created on an existing object method.

test('spies on an object method', (t) => {
  const number = {
    value: 5,
    subtract(a) {
      return this.value - a;
    },
  };
  t.mock.method(number, 'subtract');
  assert.strictEqual(number.subtract.mock.calls.length, 0);
  assert.strictEqual(number.subtract(3), 2);
  assert.strictEqual(number.subtract.mock.calls.length, 1);
  const call = number.subtract.mock.calls[0];
  assert.deepStrictEqual(call.arguments, [3]);
  assert.strictEqual(call.result, 2);
  assert.strictEqual(call.error, undefined);
  assert.strictEqual(call.target, undefined);
  assert.strictEqual(call.this, number);
});

mock.reset()

This function restores the default behavior of all mocks that were previously created by this MockTracker and disassociates the mocks from the MockTracker instance. Once disassociated, the mocks can still be used, but the MockTracker instance can no longer be used to reset their behavior or otherwise interact with them.

After each test completes, this function is called on the test context's MockTracker. If the global MockTracker is used extensively, calling this function manually is recommended.

mock.restoreAll()

This function restores the default behavior of all mocks that were previously created by this MockTracker. Unlike mock.reset(), mock.restoreAll() does not disassociate the mocks from the MockTracker instance.

mock.setter(object, methodName[, implementation][, options])

This function is syntax sugar for MockTracker.method with options.setter set to true.

Class: TestsStream

A successful call to run() method will return a new {TestsStream} object, streaming a series of events representing the execution of the tests. TestsStream will emit events, in the order of the tests definition

Event: 'test:diagnostic'

Emitted when context.diagnostic is called.

Event: 'test:fail'

Emitted when a test fails.

Event: 'test:pass'

Emitted when a test passes.

Event: 'test:plan'

Emitted when all subtests have completed for a given test.

Event: 'test:start'

Emitted when a test starts.

Class: TestContext

An instance of TestContext is passed to each test function in order to interact with the test runner. However, the TestContext constructor is not exposed as part of the API.

context.beforeEach([, fn][, options])

This function is used to create a hook running before each subtest of the current test.

test('top level test', async (t) => {
  t.beforeEach((t) => t.diagnostics(`about to run ${t.name}`));
  await t.test(
    'This is a subtest',
    (t) => {
      assert.ok('some relevant assertion here');
    }
  );
});

context.after([fn][, options])

This function is used to create a hook that runs after the current test finishes.

test('top level test', async (t) => {
  t.after((t) => t.diagnostic(`finished running ${t.name}`));
  assert.ok('some relevant assertion here');
});

context.afterEach([, fn][, options])

This function is used to create a hook running after each subtest of the current test.

test('top level test', async (t) => {
  t.afterEach((t) => t.diagnostics(`finished running ${t.name}`));
  await t.test(
    'This is a subtest',
    (t) => {
      assert.ok('some relevant assertion here');
    }
  );
});

context.diagnostic(message)

This function is used to write diagnostics to the output. Any diagnostic information is included at the end of the test's results. This function does not return a value.

context.name

The name of the test.

context.runOnly(shouldRunOnlyTests)

If shouldRunOnlyTests is truthy, the test context will only run tests that have the only option set. Otherwise, all tests are run. If Node.js was not started with the [--test-only][] command-line option, this function is a no-op.

context.signal

Warning On Node.js v14.x, this feature won't be available unless you pass the --experimental-abortcontroller CLI flag or added an external global polyfill for AbortController.

test('top level test', async (t) => {
  await fetch('some/uri', { signal: t.signal });
});

context.skip([message])

This function causes the test's output to indicate the test as skipped. If message is provided, it is included in the output. Calling skip() does not terminate execution of the test function. This function does not return a value.

context.todo([message])

This function adds a TODO directive to the test's output. If message is provided, it is included in the output. Calling todo() does not terminate execution of the test function. This function does not return a value.

context.test([name][, options][, fn])

This function is used to create subtests under the current test. This function behaves in the same fashion as the top level test() function.

Class: SuiteContext

An instance of SuiteContext is passed to each suite function in order to interact with the test runner. However, the SuiteContext constructor is not exposed as part of the API.

context.name

The name of the suite.

context.signal

Warning On Node.js v14.x, this feature won't be available unless you pass the --experimental-abortcontroller CLI flag or added an external global polyfill for AbortController.

License

MIT