NPE with preserver-order=true

[asflierl]

Hi,

using TestNG 6.2, I get the following exception from a test with multiple classes, preserver-order=true:

[testng] Exception in thread "main" java.lang.NullPointerException [testng] at org.testng.TestRunner.createClassDependencies(TestRunner.java:1182) [testng] at org.testng.TestRunner.createDynamicGraph(TestRunner.java:1079) [testng] at org.testng.TestRunner.privateRun(TestRunner.java:725) [testng] at org.testng.TestRunner.run(TestRunner.java:614) [testng] at org.testng.SuiteRunner.runTest(SuiteRunner.java:335) [testng] at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:330) [testng] at org.testng.SuiteRunner.privateRun(SuiteRunner.java:292) [testng] at org.testng.SuiteRunner.run(SuiteRunner.java:241) [testng] at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) [testng] at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) [testng] at org.testng.TestNG.runSuitesSequentially(TestNG.java:1169) [testng] at org.testng.TestNG.runSuitesLocally(TestNG.java:1094) [testng] at org.testng.TestNG.run(TestNG.java:1006) [testng] at org.testng.TestNG.privateMain(TestNG.java:1316) [testng] at org.testng.TestNG.main(TestNG.java:1280)

2 of the test classes involved use factories like this:

public class A {
  public A(T param) { ... } // parameterized constructor

  @Test
  public void someMethod() {... }

  public static class Generator {
    @Factory
    public Object[] generate() {
      return new Object[] {
        new A(...), new A(...)
      };
    }
  }
}

In the testng.yaml, the test class' name is my.package.A$Generator, maybe that's something to do with it.

I tried to use a static data provider for the constructor instead, but that threw even more horrible NPEs and other stuff.

[cbeust]

Can you provide the full test case so I can reproduce this problem? Thanks.

[cbeust]

Never mind, I think I fixed the problem, can you try the beta at http://testng.org/beta ?

Thanks.

[asflierl]

With the beta the problem persists. Sadly, I cannot disclose the code because it's proprietary company stuff. I'll try to extract a minimal test case. (I already tried but it doesn't seem easily identified.)

[asflierl]

This triggers it for me:

testng.yaml:

name: integration-tests
verbose: 2
tests:
  - name: 2.2 Features
    verbose: 2
    preserveOrder: true
    classes:
      - de.imbus.testbench.Minimal$Generator
      - de.imbus.testbench.Dummy

Minimal.java:

package de.imbus.testbench;

import static org.testng.Assert.assertEquals;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;

public class Minimal {
    private int userID;

    public Minimal(int userID) {
        this.userID = userID;
    }

    @Test(dataProvider = "someStrings")
    public void someTestMethod(String word) throws Exception {
        assertEquals((word + userID).length(), 6);
    }

    @DataProvider
    public Object[][] someStrings() {
        return new Object[][] {
            { "hello" },
            { "cruel" },
            { "world" }
        };
    }

    public static class Generator {
        @Factory
        public Object[] generator() {
            return new Object[] {
                new Minimal(1),
                new Minimal(2),
                new Minimal(3),
                new Minimal(4),
                new Minimal(5)
            };
        }
    }
}

Dummy.java:

package de.imbus.testbench;

import static org.testng.Assert.assertEquals;

import org.testng.annotations.Test;

public class Dummy {
    @Test(dataProvider = "someInts")
    public void dummyTest() throws Exception {
        assertEquals(3, 3);
    }
}

[cbeust]

How did you test? On the command line or some other way?

I'm surprised to hear you're seeing the same problem because at least the line numbers should differ, so I'm guessing you might still be using your previous testng.jar file...

[asflierl]

I've used the ant task and I'm sure I was using the jar I downloaded from your beta link. :) Also, it's still the same line number in the stack trace. Seems to be the same for issue #83

[cbeust]

I can't run your example because of:

SKIPPED: dummyTest org.testng.TestNGException: Method public void test.tmp.Dummy.dummyTest() throws java.lang.Exception requires a @DataProvider named : someInts

Can you fix it?

If I remove the dataProvider from this method, since it doesn't seem to be used, the tests pass:

PASSED: dummyTest PASSED: someTestMethod("hello") PASSED: someTestMethod("cruel") PASSED: someTestMethod("world") PASSED: someTestMethod("hello") PASSED: someTestMethod("cruel") PASSED: someTestMethod("world") PASSED: someTestMethod("hello") PASSED: someTestMethod("cruel") PASSED: someTestMethod("world") PASSED: someTestMethod("hello") PASSED: someTestMethod("cruel") PASSED: someTestMethod("world") PASSED: someTestMethod("hello") PASSED: someTestMethod("cruel") PASSED: someTestMethod("world")

Test1
Tests run: 16, Failures: 0, Skips: 0

Thanks!

[asflierl]

That's interesting. I ran the classes exactly as I quoted them here and it didn't complain about the obviously missing data provider.

What's also interesting is that in your run the Dummy test runs before the tests in Minimal, even though preserve-order should ensure the opposite.

[cbeust]

Good point about the ordering, this doesn't look right, I'll look into it. It might have to do with the fact that it's a nested class.

As for the NPE, it should be fixed in the beta (I received a test case from another user), can you try it? http://testng.org/beta

[asflierl]

I just tested it with the 6.3-beta from that link (yesterday and today morning I got a 6.2.1-beta from the link) and the NPE is gone.

Thanks a lot.

[cbeust]

Great.