In-source testing for Bun test

[itsezc]

What is the problem this feature would solve?

In source testing would be great for some use cases, this has been popularised recently by Rust's module tests and now in the JS world with Vite and Vitest, it'd be great if we can land this on Bun.

What is the feature you are proposing to solve the problem?

Reference: https://vitest.dev/guide/in-source.html

I'd personally like to see a syntax similar to how Vitest approaches it, perhaps:

// index.ts

// the implementation
export function add(...args: number[]) {
  return args.reduce((a, b) => a + b, 0)
}

// in-source test suites
if (bun.test) {
  const { it, expect } = bun.test;
  it('add', () => {
    expect(add()).toBe(0)
    expect(add(1)).toBe(1)
    expect(add(1, 2, 3)).toBe(6)
  })
}

This might go hand in hand with Bun's plans for SvelteKit & Vite integrations, as these may be necessary in order for this feature to work properly.

What alternatives have you considered?

N/A

[trnxdev]

The problem is that "bun test" runs files only with the .test.ts extension. So we have to rework that, but other than that. I think it's a neat idea, the possible workaround for now is to add environment variables/params when running bun test and checking for them.

e.g: BUN_IS_TEST=1 bun test

[Jarred-Sumner]

@itsezc while this is sometimes nice, the hard part is it forces the test runner to load every single file in the repository searching for tests to run. In an ideal world you would have tests for every file anyway, but in practice this can be pretty expensive

[ethanjdiamond]

@Jarred-Sumner it's true, but it would still be nice to have the option. My project uses Nx, so tests are only run in changed libraries, which are pretty small and quick to load.

Inline has enough advantages that it's worth considering:

• No need to keep test and src directories separate and in sync
• Easy to spot under-tested classes
• When breakpointing to debug, it's all within the same file
• Needed modules for mocking are easily visible at the top of the file
• Unit tests often add some implicit documentation to classes, would be great to have those in the same file

I'd absolutely love:

1) A way for the transpiler to ignore a block when not being run by the test-runner

The if (import.meta.vitest) block that vitest uses is nice, but nesting in a block may not be necessary. It would be awesome if there was a way to do this via directives in comments like // eslint-disable-next-line and // @ts-ignore do for eslint and tsc. If we could add a directive for bun that was something like // @bun-test, where the transpiler would ignore everything below the line when not being run by the test runner that would be amazing.

2) A way to tell the test-runner to look in .ts files.

I wouldn't want the hit of checking all .ts files to hit everyone. A bunfig setting or --inline flag on bun test seems like it would be ideal.

[ethanjdiamond]

Did a little more research on this. I haven't tested it, but I believe that we can already mimic the if (import.meta.vitest) behavior with tools that already exist. It would look like this:

• In your bunfig.toml, add a test preload script that imports bun:test and sets import.meta.buntest to an object containing the bun:test exports
• Write the tests in an if (import.meta.buntest) block that uses const { it, expect } = import.meta.buntest inside of it
• In prod, add a define config that sets import.meta.buntest to 'false', which will (in theory) transpile everything in the if (import.meta.buntest) block out since the transpiler will only see if (false) which can never resolve to true.

I believe this means that the only thing stopping in-source testing is the ability to set the glob bun test is looking for instead of the current test and spec files.

Is there any chance that change could be made?

[ryoppippi]

Also, in-suorce testing is really good when you want to test functions that you don't want to export for purposes you want to keep private.

[svallory]

For anyone interested, Here's how you can get it to work (I know it's not polite to modify other people's modules, but I like it. So shhhh! don't tell anyone! And feel free to use global.TESTING instead)

bunfig.toml

[test]
  preload = ["./bun-test-preload.ts"]

bun-test-preload.ts

declare module 'bun' {
  export const testing = true;
}

// since it is defined as a const, we need to trick typescript
(Bun as any).testing = true;

sanity.ts

if (Bun.testing) {
  console.log(`Running in-source tests for ${__filename}`);
  const { expect, test } = await import('bun:test');

  test("sanity", () => {
    expect(true).toBe(true);
    expect(false).toBe(false);
  });
}
else {
  console.log(`Not running in-source tests since Bun.testing is ${JSON.stringify(Bun.testing)}`);
}

Result...

But....

[!Caution] Trying to set the module bun:test in Bun.testing to then in the tests do something like const { expect, test } = Bun.testing Will make bun explode!

[celeryclub]

If you're looking for a very simple way to make this happen, you can check import.meta.env.NODE_ENV and see if it's test. Here's an example:

// all my functions go up here...

if (import.meta.env.NODE_ENV === "test") {
  const { test, expect } = await import('bun:test');

  test("part1", () => {
    expect(part1(input)).toBe(1);
  });

  test("part2", () => {
    expect(part2(input)).toBe(2);
  });
} else {
  console.log("part 1:", part1(input));
  console.log("part 2:", part2(input));
}

[ryoppippi]

@celeryclub thanks Is this tree-shakable? It depends on env variables so Bun bundler cannot tree-shake the test code I think

[celeryclub]

@ryoppippi Yes I think you're right. This wouldn't work for built code, only for code that's being run with Bun directly.