HaroonBadsha / js-test-driver

Automatically exported from code.google.com/p/js-test-driver
0 stars 0 forks source link

Wrong test results report when test cases functions are generated in a loop #372

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
Please consider the test code:

var validatorsTests = {
    notnull: [
        [null, false],
        ['', false],
        [' ', true],
        ['string', true],
        [0, true],
        [1, false] // the only failing case
    ]
};

var testCase = TestCase('validators-test');

for (var validatorName in validatorsTests) {
    var examplesList = validatorsTests[validatorName];
    for(var i in examplesList) {
        var example = examplesList[i];
        var input = example[0];
        var expected = example[1];
        var testName = 'test validator ' + validatorName + ' when ' + typeof(input) + '(' + String(input) + ') given';

        testCase.prototype[testName] = function() {
            assertTrue(JastinValidator[validatorName]() === expected);
        }
    }
}

testCase.prototype.testHehe = function() {
    assertTrue(true);
}

And the source code:

var JastinValidator = {
    'notnull':function (value, element) {
        return value !== null && String(value).length > 0;
    }
}

This test suite works but the reports are bad. If any dynamicaly generated test 
fails, test results report that EVERY dynamic test failed. You probably look at 
the stacktrace which is the same for every test, instead of the test name, to 
tell if the test succeeded or failed?

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

Original issue reported on code.google.com by enwukaer@gmail.com on 14 May 2012 at 2:38

GoogleCodeExporter commented 8 years ago
There should be
  assertTrue(JastinValidator[validatorName](input) === expected);
instead of
  assertTrue(JastinValidator[validatorName]() === expected);

Just made a mistake when preparing a test to paste here.

Original comment by enwukaer@gmail.com on 14 May 2012 at 3:12

GoogleCodeExporter commented 8 years ago
Please review how closures work. You are setting all the tests to the last test.

Original comment by corbinrs...@gmail.com on 14 May 2012 at 5:53

GoogleCodeExporter commented 8 years ago
Thanks for pointing it out! My friend helped me with the closures. The code 
should look like this:

        testCase.prototype[testName] = function (validatorName, expected) {
            return function () {
                console.log(validatorName);
                assertTrue(JastinValidator[validatorName](input) === expected);
            }
        }(validatorName, expected);

Original comment by enwukaer@gmail.com on 14 May 2012 at 6:42