search

avajs / ava

Node.js test runner that lets you develop with confidence 🚀
MIT License
20.74k stars 1.41k forks source link

Report bad test() usage as a test failure, rather than a runtime error #698

Open jamestalmage opened 8 years ago

jamestalmage commented 8 years ago

Issuehunt badges

Sometimes you get sloppy / distracted and end up with a half completed test:

// you have some good tests:

test('foo', t => {
  // this will never be executed because of the mistakes below
});

// This throws - saying it should be a todo test:
test('fooo');

// Ooops - did you get distracted?:
test();

// Todo should not have an implementation, it is intended to document future tests/features.
test.todo('foo', t => {});

Instead of throwing when someone misuses the API, I think the misuse should just be reported as a failure. It should still execute the correctly written tests.

There is a $40.00 open bounty on this issue. Add more on Issuehunt.

sindresorhus commented 8 years ago

I like the idea of being as graceful a possible. One concern though. Look at it the other way around. What if you have a lot of tests and you use one of them incorrectly. Now it will run all the tests before letting you know you have an issue and then you'll have to rerun the whole test file again. Just a thought.

novemberborn commented 8 years ago

78 will likely force our hand here. We're doing #696 so we can correctly run exclusive test but I doubt we'll try and suss out these issues statically. If we don't then we'll end up in situations where we're already running a whole bunch of tests before the fault is encountered. Then why not run to completion?

jamestalmage commented 8 years ago

Now it will run all the tests before letting you know you have an issue and then you'll have to rerun the whole test file again. Just a thought.

No reason we can't report it as an error as soon as it's encountered.

novemberborn commented 8 years ago

No reason we can't report it as an error as soon as it's encountered.

You probably won't notice it though.

jamestalmage commented 8 years ago

Why not? If it increases the error count, that's noticeable immediately.

novemberborn commented 8 years ago

Why not? If it increases the error count, that's noticeable immediately.

Fair enough.

novemberborn commented 6 years ago

This will be easiest to implement once https://github.com/avajs/ava/pull/2217 lands.

IssueHuntBot commented 5 years ago

@issuehunt has funded $40.00 to this issue.