mocha-parallel / mocha-parallel-tests

Parallel test runner for mocha tests. Looking for maintainer.
MIT License
200 stars 45 forks source link

Dynamic tests #177

Closed tosbaha closed 6 years ago

tosbaha commented 6 years ago

I have some dynamic tests in a single file and it seems mocha-parallel-tests doesn't introduce any difference if test is run from single file. Is there a any workaround to speed up dynamic tests in a single file? For example below script takes 14 seconds to run with mocha or mocha-parallel-tests. However if I change describe with parallel from mocha.parallel it takes 2 seconds. I don't want to use parallel because I don't want to fire up lots of requests all once and face socket limit problem.

const objects = [
    {"name":"Test1","site":"http://www.site1.com"},
    {"name":"Test2","site":"http://www.site2.com"},
    {"name":"Test3","site":"http://www.site3.com"},
    {"name":"Test4","site":"http://www.site4.com"},
    {"name":"Test5","site":"http://www.site5.com"},
    {"name":"Test6","site":"http://www.site6.com"},
    {"name":"Test7","site":"http://www.site7.com"},
];

 function doWork(done){
    setTimeout(done,2000);
}

describe('Test me',function () {
    objects.forEach(function (object) {
        it(`${object.name} should test`, function (done) {
            this.timeout(5000);
            console.log(`Testing ${object.site}`)
            doWork(done);
        });
    });
});
1999 commented 6 years ago

Well, for this kind of tests there would definitely be difference because what mocha-parallel-tests does is it;s running each file in a separate process. Perhaps if you split your file into separate files you can see the difference.

Using mocha.parallel is okay if you don't need all mocha machinery: reporters, CLI, etc: https://github.com/yandex/mocha-parallel-tests/wiki/Comparison-with-mocha.parallel

tosbaha commented 6 years ago

That is a bummer :\ My project needs dynamic tests because it would be tedious to write 300 test and maintain it. I didn't understand what you mean by

if you don't need all mocha machinery: reporters, CLI, etc

Because if I run the test by mocha using parallel, I can use the reporters like mochawesome. Only downside is, because parallel must be the top level function, I can't separate each individual tests with describe.

PS: I have found a way to run tests in chunks with parallel. It seems to work and gives me flexibility to change the chunk size.

const _ = require('lodash');

const objects = [
    {"name":"Test1","site":"http://www.site1.com"},
    {"name":"Test2","site":"http://www.site2.com"},
    {"name":"Test3","site":"http://www.site3.com"},
    {"name":"Test4","site":"http://www.site4.com"},
    {"name":"Test5","site":"http://www.site5.com"},
    {"name":"Test6","site":"http://www.site6.com"},
    {"name":"Test7","site":"http://www.site7.com"},
];
const chunks = _.chunk(objects,4);

 function doWork(done){
    setTimeout(done,2000);
}

function singleTest(object) {
        it(`${object.name} should test `, function (done) {
            this.timeout(5000);
            console.log(`Testing ${object.site}`);
            doWork(done);
         });
}

function smallTests(chunk) {
    parallel('Tests', function() {
        chunk.forEach(function (object) {
            singleTest(object);
        });
    });
}

     chunks.forEach(function (chunk) {
         smallTests(chunk)
     })

Thanks again for the insight. You may close the issue.

1999 commented 6 years ago

Because if I run the test by mocha using parallel, I can use the reporters like mochawesome.

Interesting. Can you give me an example of that? I didn't find a way to use custom reporters with mocha.parallel

tosbaha commented 6 years ago

I haven't done anything special. I put my test file inside test folder and run the test with mocha -R mochawesome It works. Only downside as I said is, test doesn't show how it passed. Mochawesome only reports resolution of parallel

Just paste above into /test/ folder and run it with mocha -R mochawesome

Test Results with parallel

Duration of all tests: 4.17s

screen shot 2018-06-27 at 15 42 13

Test Results with describe

Duration of all tests: 14.22s

screen shot 2018-06-27 at 15 46 18
1999 commented 6 years ago

Gotcha. That's an interesting example of using mocha.parallel: it() comes from mocha which patches the global namespace while parallel() is just a function from mocha.parallel. I didn't think of it before 👍