krieselreihe / natr

⚠️WIP: Not another test runner! But it is, and it is a highly opinionated one.
MIT License
0 stars 0 forks source link

Async stream tests create internal problem #5

Open MartinHelmut opened 5 years ago

MartinHelmut commented 5 years ago

There are async tests failing with no obvious reason behind. A @todo comment was placed here: https://github.com/krieselreihe/natr/blob/master/src/__tests__/duplex-stream.test.js#L36

The following both tests kind of result in a timeout internally that will result in setting the test suite pointer to the next item and therefore adding the result messages to the wrong queue.

Wrong test output:

TAP version 13
# duplex streams
ok 1 - Given default data: should return the input stream
# regular test cases
ok 2 - Given the calc function without arguments: should return 0
ok 3 - Given the calc function with one argument: should return the given number
# ...

Correct test output:

TAP version 13
# duplex streams
ok 1 - Given an input stream: should be possible to be paused
ok 2 - Given undefined data: should not stop if written
ok 3 - Given an input stream: should end before close
ok 4 - Given an input stream with auto destroy deactivated: should stay open
# ...

Here the tests that cause the issue:

describe("duplex streams", async assert => {
  // 100 is just a generic length for the test data set
  const expected = Array.from({ length: 100 }, () =>
    Math.floor(Math.random() * 100)
  );

  assert({
    given: "default data",
    should: "return the input stream",
    actual: await execute(() => {
      const stream = duplexStream();
      const mockedStream = spec(stream)
        .through()
        .pausable();
      const result = new Promise((resolve, reject) => {
        read(stream, (err, actual) => {
          if (err) reject(err);
          resolve(actual);
        });
      });

      stream.on("close", mockedStream.validate);
      write(expected, stream);

      return result;
    }),
    expected
  });

  assert({
    given: "custom data",
    should: "returns the input stream",
    actual: await execute(() => {
      const stream = duplexStream(data => stream.emit("data", data * 2));
      const mockedStream = spec(stream)
        .through()
        .pausable();

      const result = new Promise((resolve, reject) => {
        read(stream, (err, actual) => {
          if (err) reject(err);
          resolve(actual);
        });
      });

      stream.on("close", () => {
        mockedStream.validate();
      });
      write(expected, stream);

      return result;
    }),
    expected: expected.map(data => data * 2)
  });
});
MartinHelmut commented 4 years ago

An easier example of this behaviour is the following test suite:

// src/__tests__/long-running.test.js
const { describe, execute } = require("../runner");

describe("with long running unit tests", async (assert) => {
  assert({
    given: "first test for long running tests",
    should: "execute before 'a long running unit test'",
    actual: true,
    expected: true,
  });

  assert({
    given: "a long running unit test",
    should: "resolve in order to the other tests",
    actual: await execute(() => {
      return new Promise((resolve) => {
        setTimeout(() => resolve(true), 1000);
      });
    }),
    expected: true,
  });

  assert({
    given: "second test for long running tests",
    should: "execute after 'a long running unit test'",
    actual: true,
    expected: true,
  });
});