group-by-instances is not working as expected

TestNG Version 7.0.0

Created 2 data provider and test methods test1 and test2. Test2 is depend on test1.

package com.automation.test;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class GroupByInstances {

    @Test(dataProvider="testdp1",priority=2)
    public void test1(String str) {
        System.out.println(str);
    }

    @Test(dataProvider="testdp2",dependsOnMethods= {"test1"},priority=1)
    public void test2(String str) {
        System.out.println(str);
    }

    @DataProvider(name="testdp1")
    public Object[][] testdb1() {
           return new Object[][] { { "Cedric" }, { "Beust" } }; 
    }

    @DataProvider(name="testdp2")
    public Object[][] testdb2() {
          return new Object[][] { { "Rajiv" }, { "Sharma" } };  
    }
}`

TestNG xml file

<suite name="Suite" group-by-instances="true">
    <test name="Test" group-by-instances="true">
        <classes>
            <class name="com.automation.test.GroupByInstances"></class>
        </classes>
    </test>
</suite>

Expected behavior

Cedric Rajiv Beust Sharma

Actual behavior

Cedric Beust Rajiv Sharma

Also tried with Factory class but same result I got-:

 public class GroupByInstancesFactory {

    @Factory
    public Object[] testFactory() {
        return new Object[] {new GroupByInstances()};
    }
}

TestNG xml for Factory

<suite name="Suite" group-by-instances="true">
    <test name="Test" group-by-instances="true">
        <classes>
            <class name="com.automation.test.GroupByInstancesFactory"></class>
        </classes>
    </test>
</suite>

Results -: Cedric Beust Rajiv Sharma

Expected -: Cedric Rajiv Sharma Beust

Is the issue reproductible on runner?

[x] Maven
[x] Eclipse

You have a wrong understanding of the working of group-by-instances attribute.

It is applicable only when your @Factory annotated method produces two or more test class instances and when you want all the @Test methods in each of those test class instance to run together.

Your sample wasn't doing any of that.

Now coming to ask, I dont think TestNG will support what you are asking for which is "A data driven test method that depends on another data driven test method needs to have interleaved executions wherein the dependencies are honored on a per iteration basis".

TestNG does not support this execution model.

TestNG will mark a test method as done only if all the iterations of it have run to completion before it runs the dependent methods.

You can try circumventing around this, by doing something like this

Closing this issue with resolution as Working as designed

import java.util.Arrays;
import java.util.List;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;

public class TestClassInstance {

  private List<String> instances;

  @Factory(dataProvider = "masterTestData")
  public TestClassInstance(List<String> instances) {
    this.instances = instances;
  }

  @Test(dataProvider = "testClassTestData")
  public void testMethod(String data) {
    System.err.println("Instance:" + hashCode() + " data: " + data);
  }

  @DataProvider(name = "testClassTestData")
  public Object[][] testClassTestData() {
    Object[][] data = new Object[instances.size()][1];
    for (int i = 0; i < instances.size(); i++) {
      data[i][0] = instances.get(i);
    }
    return data;
  }

  @DataProvider(name = "masterTestData")
  public static Object[][] masterTestData() {
    return new Object[][] {{Arrays.asList("Cedric", "Rajiv")}, {Arrays.asList("Beust", "Sharma")}};
  }
}

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="2184_suite" parallel="false" verbose="2" group-by-instances="true">
  <test name="2184_test">
    <classes>
      <class name="com.rationaleemotions.github.issue2184.TestClassInstance"/>
    </classes>
  </test>
</suite>

...
... TestNG 7.0.0 by CÃ©dric Beust (cedric@beust.com)
...

Instance:410495873 data: Cedric
Instance:410495873 data: Rajiv
Instance:811587677 data: Beust
Instance:811587677 data: Sharma

PASSED: testMethod("Cedric")
PASSED: testMethod("Rajiv")
PASSED: testMethod("Beust")
PASSED: testMethod("Sharma")

===============================================
    2184_test
    Tests run: 4, Failures: 0, Skips: 0
===============================================

===============================================
2184_suite
Total tests run: 4, Passes: 4, Failures: 0, Skips: 0
===============================================

Process finished with exit code 0

testng-team / testng