Wrong test results report when test cases functions are generated in a loop

[GoogleCodeExporter]

Please consider the test code:

var validatorsTests = {
    notnull: [
        [null, false],
        ['', false],
        [' ', true],
        ['string', true],
        [0, true],
        [1, false] // the only failing case
    ]
};

var testCase = TestCase('validators-test');

for (var validatorName in validatorsTests) {
    var examplesList = validatorsTests[validatorName];
    for(var i in examplesList) {
        var example = examplesList[i];
        var input = example[0];
        var expected = example[1];
        var testName = 'test validator ' + validatorName + ' when ' + typeof(input) + '(' + String(input) + ') given';

        testCase.prototype[testName] = function() {
            assertTrue(JastinValidator[validatorName]() === expected);
        }
    }
}

testCase.prototype.testHehe = function() {
    assertTrue(true);
}

And the source code:

var JastinValidator = {
    'notnull':function (value, element) {
        return value !== null && String(value).length > 0;
    }
}

This test suite works but the reports are bad. If any dynamicaly generated test 
fails, test results report that EVERY dynamic test failed. You probably look at 
the stacktrace which is the same for every test, instead of the test name, to 
tell if the test succeeded or failed?

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

AssertError: expected true but was false
()@http://localhost:9876/test/src/test/resources/jstestdriver/validators-test.js
:23

Original issue reported on code.google.com by enwukaer@gmail.com on 14 May 2012 at 2:38

[GoogleCodeExporter]

There should be
  assertTrue(JastinValidator[validatorName](input) === expected);
instead of
  assertTrue(JastinValidator[validatorName]() === expected);

Just made a mistake when preparing a test to paste here.

Original comment by enwukaer@gmail.com on 14 May 2012 at 3:12

[GoogleCodeExporter]

Please review how closures work. You are setting all the tests to the last test.

Original comment by corbinrs...@gmail.com on 14 May 2012 at 5:53

• Changed state: Invalid

[GoogleCodeExporter]

Thanks for pointing it out! My friend helped me with the closures. The code 
should look like this:

        testCase.prototype[testName] = function (validatorName, expected) {
            return function () {
                console.log(validatorName);
                assertTrue(JastinValidator[validatorName](input) === expected);
            }
        }(validatorName, expected);

Original comment by enwukaer@gmail.com on 14 May 2012 at 6:42