search

ben-page / node-test

A simple, asynchronous test runner for Node.js.
https://www.npmjs.com/package/node-test
MIT License
4 stars 1 forks source link

A few questions ... #1

Closed ghost closed 8 years ago

ghost commented 8 years ago

Hey! Great project! Just a few questions:

ghost commented 8 years ago

I extend this list with questions...

ghost commented 8 years ago

@ben-page Any thoughts about a more familiar API with the Tape family? I wonder why wont to force the end-user to create a new instance in each new test they create?

Look at one of the first commit for AVA. Similiar code as you have, and it's already done there. Also your hooks are done differently with events. Any plans for this?

ben-page commented 8 years ago

when will you add a additonal reporter?

I don't have plans to add additional reporters. But I will probably implement the js-reporter interface so that other reporters can easily be used.

any log options as in ava, tape etc?

Could you be more specific? I'm not sure what you are referring to.

are each process isolated from each other so you can change global state or overriding a built-in in one test file, without affecting another?

No, currently all tests/suites are run in the same process. One of the core principles of the project is to be able to simple run a test file and debug it like any other node js file. I think automatically running tests in different processes/context violates this principle.

However, I think it makes sense for the CLI (which doesn't exist, yet) to have this ability.

any plans to support serials as in ava?

Serials are already supported.

option for plugins? Keep the core small and stable as possible, and let the user extend it with plugins to extend various part of the code?

Not yet. I like this idea in theory. I'm not yet sure what the plugins would do or where to provide hooks for them.

coverage report support? Is this spwaned too, so NYC need to be user?

I'm not sure the value of integrating a code coverage tool into this. You can just run them separately. However, you can run anything you want inside a test.

can't see any direct support for windows. E.g. Appveyor. Any plans?

I develop in Windows and Linux. So, it works in Windows. Not sure what is need for appveyor support, but feel free to open a bug report about any issues you have.

an end-dev wont to have thing simple. Is it possible to create a suite inline inside the code, and avoid the end-dev doing it? It will speed up the way of creating tests.

Hmm, I'll need to think on this. My immediate reaction is that mixing tests into you code is a pretty bad idea. Can you provide an example of someone doing this well?

what about adding in support for testing node modules? Maybe inherit a few features from teenytest?

What do you mean by "testing node modules"? Do you mean running all of your tests at once via "npm test" or something like that?

I typically do that by creating an index test file. For example, here's the contents of the test directory of one of my projects:

find.js
foreign-keys.js
index.js
observable.js
sync.js
types.js

index.js looks like this

'use strict';
require('./find');
require('./foreign-keys');
require('./observable');
require('./sync');

Then in my package.json, I have

  "scripts": {
    "test": "node test/index.js"
  }

I do plan to create a CLI to make that easier. But for now, I'm ok with this solution.

ben-page commented 8 years ago

a single glob or an array with globs? What is supported? Support multiple tests at once would be nice. test1.js test3.js test5.js or [test1.js test3.js test5.js].

Yep, CLI is coming eventually

is there any validations in this code? Error tracking? E.g. you try to test a invalid glob.

No, if your tests are invalid, they'll throw error and you can fix them.

BDD or TDD? Both supported, or will be supported?

I think it's fair to call this existing interface TDD. I don't plan to add other interfaces. TDD vs BDD is really about how you design your tests, not about the names of your functions (describe() vs test()).

ES2015 & ES2016 suported out of the box, or is there a kind of require option as in Mocha?

I started this project in July of 2016 and ES2015 is the standard. I don't have a the desire or need to support old environments.

It may be worth supporting testing in older browsers. Most likely I would add support for running a suite in a browser via Selenium.

How to exclude some files? Plans to add support for it?

I don't know what you mean.

TypeScript supported out of the box?

I don't use TypeScript. I have no idea if it works.

plans to add timeout support?

Yes

highlights slow tests?

Yes

any kind of grep feature planned?

For what?

delay option for async?

What value would this be? Will you explain what you would use this for?

any options to activate use of all Harmony features ?

Are you referring to the non-existent CLI again?

plan to add a option to set number of times to retry failed test cases?

No. I think this is a bad design. Tests should be deterministic. And if you absolutely require this feature, you can easily build it inside the test.

ben-page commented 8 years ago

@ben-page Any thoughts about a more familiar API with the Tape family? I wonder why wont to force the end-user to create a new instance in each new test they create?

What do you mean by more familiar API? Do you mean plan()? If so, then absolutely not. That approach is cumbersome and error prone.

Look at one of the first commit for AVA. Similiar code as you have, and it's already done there. Also your hooks are done differently with events. Any plans for this?

Which commit? What code? My hooks are not events. I really have no idea what you mean.

ghost commented 8 years ago

I liked you replies, and the way you are going with this code! Awsome stuff!

As a reply I choose to start with plugins. Ava have build-in Babel support, but block itself from other files and languages such as coffe script, typescript etc.

A plugin could simply let the end-dev create it's own plugin to handle this languages. More or less the same way as you see in Karma's preprocessor. This will not bloat your core, and will be up to end-dev what too use and how to use it. Then you would support all languages.

Mocha have a require flag in the CLI to handle this, but as I understand you this code should work both for CLI and node? A plugin like this will solve both.

For the more familiar API. Today when you write a test, you have to create a new Suite instance within the test itself. Is that needed? A easier and more familier API would be not to do that. Just include the file, and in the main source file you create a new instance.

Example:

return function(options) {
// deal with the options here. E.g. plugin etc.
let s = new Suite;
return function(name, cb) {
// add tests here
}
}

// In the tests...

suite('test...', function(t) {} '); // no suite.test needed

If you need to hook the hooks on it, it could be done this way in the main source file

let x = Suite.test(); // main
x.before = s.before()..
x.after = s.after()..

return x;

Just dummy code to illustrate how I'm thinking. But in this way you would have same thing as Tape and Ava.

Agree with you regarding plan(). I can't see the point with it either.

Regarding excluding files. If you support Glob you can simply do this:

src/{server/*,server/!(test|coverage)/**}.js

But this will only work for the CLI. So if you have an option to define which files to exclude it will work in the same way. That was my thinking.

E.g. Istanbul require some files to be excluded from the tests to work. Take a look at how you exclude files in Karma or Mocha. Same thing.

Testng node modules...

// node module
let abc = export default {
test1: function() {},
test2: function() {}
test3: function() {}
}

You import the node module and not the file itself, and if you want to test test2 in that module you could do this:

abc.test2  // like any objects

Again a silly example.

Last thing here is your reporter. It's a good thing to integrate js-report support. And that could also be plugin material. Allow various plugin created.

Your current reporter I hope you plan to improve it? E.g. look at the chalk NPM package.

A log function will just help you keep track of things when you debugging. See Karma as the best example on logging.

And why not use native ES6 promises instead of Bluebird? Bluebird is a commonJS module and if you develop for ES2015 it will be tricky to import all it's stuff. It's not an ES module.

ghost commented 8 years ago

Regarding harmony. Yes CLI. A boolean flag to turn it on or off. E.g. Istanbul is more happy with harmony activated, but some NodeJS versions isn't so happy. E.g. iron-node.

ben-page commented 8 years ago

API

I think the current approach makes things very flexible at the cost of only a small amount of verbosity. I don't want to impose much structure on the user. They should be free to include multiple suites in a single file or in multiple files.

Plugins

I think this is good idea, but I don't have plans to work on it right now.

CLI

File globs are definitely part of the plan for the CLI.

Reporter

Adding color to the report is a good idea. Other than that, I don't plan to change the output much.

ES6 Modules

I'll support ES6 modules when Node.js does. I don't plan to work on them until the spec is complete and Node.js supports them.

Promises

Compared to ES6 Promises, Bluebird is more reliable, faster, and easier to use. Namely, ES6 Promises can swallow Errors in several places and Bluebird provides several extremely helpful extensions to A+ Promises.

ghost commented 8 years ago

@ben-page Any ideas when you are going to add js-reporter support, and EventEmitter? And what is the roadmap for the CLI ?

ben-page commented 8 years ago

Real life is taking priority right now. I'll try to find time for it the next couple weeks.