mystery fail with async test

mixmix commented 2 years ago

Describe the issue

We see a test file which when run in isolation works, but fails when run with a glob script like

    "test:js": "cross-env NODE_ENV=test tape -r esm 'test/**/*.test.js' | tap-arc",

The error looks like:

test count(1) != plan(null)

The test looks like :

const test = require('tape')
const TestBot = require('./test-bot')

function setENV (env) {
  process.env.NODE_ENV = env
  // console.info('set NODE_ENV=', env)
}

// this test passes in isolation, but fails in the suite... why?
// this only happens with tap-arc T_T
// tap-spec works fine
test('init (run migrations)', t => {
  t.plan(1)

  const initialENV = process.env.NODE_ENV
  setENV('production')

  TestBot()
    .then(({ ssb }) => {
      setENV(initialENV)
      t.pass('ran migrations, started up')
      ssb.close()
    })
    .catch(err => t.error(err, 'init (run migrations) threw an error')) // should not be run
})

Inside TestBot there is some async stuff running database migrations, a setImmediate What's strange is tap-spec doesn't hit this bug, and tape without a pretty-printer is fine.

REPO: https://gitlab.com/ahau/lib/ssb-plugins/ssb-ahau file : tests/init.test.js

tbeseda commented 2 years ago

Yikes. Thanks for reporting and the reproduction steps. I wonder if it's related to this bug https://github.com/architect/tap-arc/issues/28 And either way I think both are related to how tap-arc handles streams. I need to iron that out, and as mentioned previously, it has generally outgrown its single-file design and could use a refactor.

Things are a bit busy in other areas, but I'm hoping to carve out some time for this lib soon.

mixmix commented 2 years ago

No hurry from me - I just wanted to document what little hints i found. I tried writing a simpler test but it didn't work.

I wondered if it was timing out on the test too early? dunno

architect / tap-arc

mystery fail with async test #30