Incorrect hierarchy in testng.xml file created programmatically and failed to run.

[HariHaraBhuvanesh]

TestNG Version

7.0.0 (my current version but issue persists in all versions)

I have a requirement to create custom testng.xml file based on the inputs. I was able to create one successfully but the groups in the suite level are misplaced i.e below the parameters and the listeners instead of the top of the parameters i.e groups should be placed first and then the others as specified in the below error.

The content of element type "suite" must match "(groups?,(listeners|packages|test|parameter|method-selectors| suite-files)*)".

Below is piece of code to create the xml file.

        XmlSuite xmlSuite = new XmlSuite();
        xmlSuite.setName("CustomSuiteFile");
        xmlSuite.setParallel(ParallelMode.TESTS);
        xmlSuite.setThreadCount(1);
        List<String> group = new ArrayList<String>();
        group.add("Regression");
        group.add("Smoke");
        xmlSuite.setIncludedGroups(group);
        xmlSuite.setListeners(Arrays.asList("Utils.TestListener"));
        parameters.put("reportPath", "C://ExtentReports//ExtentReportResult");
        xmlSuite.setParameters(parameters);

Expected behavior

the group is placed first and then the others in the suite level.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite thread-count="1" name="CustomSuiteFile" parallel="tests">
<groups>
    <run>
      <include name="Regression"/>
      <include name="Smoke"/>
    </run>
  </groups>
  <parameter name="reportPath" value="C://ExtentReports//ExtentReportResult"/>
  <listeners>
    <listener class-name="Utils.TestListener"/>
  </listeners>
  <test thread-count="1" name="Chrome" parallel="tests">
    <parameter name="browser" value="Chrome"/>
  </test>
</suite>

Actual behavior

the group is placed below the listener which is causing the issue.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite thread-count="1" name="CustomSuiteFile" parallel="tests">
  <parameter name="reportPath" value="C://ExtentReports//ExtentReportResult"/>
  <listeners>
    <listener class-name="Utils.TestListener"/>
  </listeners>
  <groups>
    <run>
      <include name="Regression"/>
      <include name="Smoke"/>
    </run>
  </groups>
  <test thread-count="1" name="Chrome" parallel="tests">
    <parameter name="browser" value="Chrome"/>
  </test> <!-- Chrome -->
</suite> <!-- CustomSuiteFile -->

Is the issue reproductible on runner?

• [ ] Shell
• [x] Maven
• [ ] Gradle
• [ ] Ant
• [x] Eclipse
• [x] IntelliJ
• [x] NetBeans

[krmahadevan]

@HariHaraBhuvanesh - How does the order of elements in the suite xml file affect your execution ?

[krmahadevan]

@HariHaraBhuvanesh - Never mind. I got the issue. So it looks like you are basically using the TestNG apis to generate a suite xml file and then feeding it to the TestNG suite xml parsing mechanism which is where your code is failing.

Below sample reproduces the entire problem

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.testng.TestListenerAdapter;
import org.testng.annotations.Test;
import org.testng.xml.Parser;
import org.testng.xml.XmlClass;
import org.testng.xml.XmlSuite;
import org.testng.xml.XmlSuite.ParallelMode;
import org.testng.xml.XmlTest;

public class IssueTest {

  @Test
  public void issueTest() throws IOException {
    XmlSuite xmlSuite = new XmlSuite();
    xmlSuite.setName("CustomSuiteFile");
    xmlSuite.setParallel(ParallelMode.TESTS);
    xmlSuite.setThreadCount(1);
    List<String> group = Arrays.asList("Regression", "Smoke");
    xmlSuite.setIncludedGroups(group);
    xmlSuite.addListener(TestListenerAdapter.class.getName());
    Map<String, String> parameters = new HashMap<>();
    parameters.put("reportPath", "C://ExtentReports//ExtentReportResult");
    xmlSuite.setParameters(parameters);
    XmlTest xmlTest = new XmlTest(xmlSuite);
    xmlTest.setName("Chrome");
    xmlTest.setXmlClasses(Collections.singletonList(new XmlClass(ExampleTestCase.class.getName())));
    File file = File.createTempFile("sample", ".xml");
    Files.write(file.toPath(), xmlSuite.toXml().getBytes());
    Collection<XmlSuite> suites = new Parser(file.getAbsolutePath()).parse();
    suites.forEach(xmlSuite1 -> System.err.println(xmlSuite1.toXml()));
  }

  public static class ExampleTestCase {
    @Test
    public void testMethod() {}
  }
}

[HariHaraBhuvanesh]

@krmahadevan

Yes exactly I tried the same way you did. Since the groups are placed in the last position at suite level it throws the error. Per my understanding, groups should be placed first and then other details so that execution would work without any issues.

[HariHaraBhuvanesh]

@krmahadevan

is this fixed ? bug is still reproducible. I updated to version 7.1.0 as well but the xml is not generated as expected. Or should i wait until next release. Please advise.

[krmahadevan]

This is fixed and will be available in TestNG 7.2.0 when it gets released.