Documentation suggests test.todo always runs tests and fails if they pass

What is the type of issue?

Documentation is incorrect

What is the issue?

test.todo was added in 0.6.3 and failed if the test passed:

// a.test.js
test.todo("todo test still runs", () => {
  expect(0).toBe(0);
});

$> bun-0.6.3 wiptest a.test.js
bun wiptest v0.6.7 (59d7c47e)

a.test.js:
✗ todo test still runs [0.10ms]
  ^ this test is marked as todo but passes. Remove `.todo` or check that test is correct.

 0 pass
 1 fail
 1 expect() calls
Ran 1 tests across 1 files. 1 total [4.00ms]

Behaviour was updated in 0.6.6 to only run todo tests if the --todo flag was passed, and it is still that way now in the latest version of bun:

$> bun-0.6.6 wiptest a.test.js
a.test.js:
✎ todo test still runs

 0 pass
 1 todo
 0 fail

$> bun-0.6.6 wiptest --todo a.test.js
a.test.js:
✗ todo test still runs
  ^ this test is marked as todo but passes. Remove `.todo` or check that test is correct.

 0 pass
 1 fail
 1 expect() calls

https://bun.sh/docs/test/writing#test-todo:

Mark a test as a todo with test.todo. These tests will be run, and the test runner will expect them to fail. If they pass, you will be prompted to mark it as a regular test.

todo tests only run with the --todo flag, this is wrong.

To exclusively run tests marked as todo, use bun test --todo.

--todo still runs non-todo tests, this is wrong

https://bun.sh/guides/test/todo-tests:

If an implementation is provided, it will be executed and expected to fail by test runner! If a todo test passes, the bun test run will return a non-zero exit code to signal the failure

todo tests only run with the --todo flag, this is wrong. Interestingly, the guide was written after the change in 0.6.6 and was wrong at the time of writing.

I don't really see the point of test.todo if they only run with a flag --todo that no one uses, they're basically just test.skip with a different name

Where did you find it?

No response

oven-sh / bun