Lazy initialization doesn't work when using Factory with lazy DataProvider

[leo-13]

Steps to reproduce:

• Create a test class MyTest.java:

import org.testng.annotations.*;

public class MyTest {
    private String name;

    @DataProvider
    public static java.util.Iterator generateTestsLazy() {
        return new MyIterator();
    }
    @Factory(dataProvider = "generateTestsLazy")
    public MyTest(String a){
        name = a;
        System.out.println("Creating instance for " + name);
    }

    @BeforeClass
    public void setUp(){
        System.out.println("Setup test: " + name);
    }
    @Test
    public void runTest(){
        System.out.println("Running test: " + name);
    }
}

• Create Iterator class MyIterator.java:

import java.util.Iterator;

public class MyIterator implements Iterator {
    Object[][] names = {
          new Object[] { "test1"} ,
          new Object[] { "test2"} ,
          new Object[] { "test3"} ,
    };
    private int index = 0;

    public boolean hasNext() {
        return index < names.length;
    }
    public Object next() {
        Object[] result = names[index];
        index++;
        return result;
    }
    public void remove() {
        throw new UnsupportedOperationException();
    }
}

• Run test suite with group-by-instances="true"

  Actual result:

No lazy initialization. All instances are created before the tests are started:

Creating instance for test1 Creating instance for test2 Creating instance for test3 Setup test: test3 Running test: test3 Setup test: test2 Running test: test2 Setup test: test1 Running test: test1

Expected result:

Lazy initialization looking like this: Creating instance for test1 Setup test: test1 Running test: test1 Creating instance for test2 Setup test: test2 Running test: test2 Creating instance for test3 Setup test: test3 Running test: test3

[leo-13]

Another issue here is that I see first constructor call prior to DataProvider call. Why could this happen?

[adamjson]

I am also running into this issue. Are there any plans to fix this?

[adamjson]

Lazy initialization using an iterator works perfectly when the data provider is specified in a Test annotation, but NOT when it is specified in a Factory annotation.

[juherr]

The current architecture of testng create all instances, collect tests, order them and run them.

To fix this issue, we will have to change a lot, what is not planned yet.

[BrandonDudek]

+1

This is needed!