Priority on configuration methods

[dianny123]

TestNG Version

Latest 7.1.2-SNAPSHOT

Expected behavior

Today only the annotation test support priority , but the priority for all configuration methods is needed as well. If there are multi methods for the same type of configuration method in the same test case, these methods could execute in the order of priority.

Is the issue reproductible on runner?

New feature requirement.

Test case sample

ConfigurationMethodPriorityTest.java

@BeforeMethod(priority=1)
public void testFirst(){
}

@BeforeMethod(priority=2)
public void testSecond(){
}

@BeforeMethod(priority=3)
public void testThird(){
}

The execution order will be testFirst() testSecond() testThird()

Other type of configuration methods work as this mechanism。

[krmahadevan]

@cbeust @juherr What are your thoughts on this ?

[juherr]

I understand the feature but I'm not sure about the way to do it: @BeforeMethod(priority=3) could mean "run only before method with priority 3"

A basic approach will be more like:

public void testFirst(){
}

public void testSecond(){
}

@BeforeMethod
public void testThird(){
  testFirst();
  testSecond();
  ...
}

If the need is to mark steps, I think there is some 3rd lib for that purpose.

[dianny123]

Thanks for your reply. The above test case is a simple example. Here is a sophisticated scenario. If the multi configuration methods are declared belong to different groups, then specific the groups which to be run, the methods which belong to this group will execute in the order of priority. Code example is:

@BeforeMethod(groups={“functest”, "checkintest"})(priority=0)
public void testFirst(){
}

@BeforeMethod(groups={“functest”, "performance"})(priority=1)
public void testSecond(){
}

@BeforeMethod(groups={“checkintest”, "performance"})(priority=2)
public void testThird(){
}

@BeforeMethod(groups={“functest”})(priority=2)
public void testForth(){
}

@BeforeMethod(groups={"checkintest"})(priority=3)
public void testFifth(){
}

If run group "functest", the execute order will be:

testFirst()
testSecond()
testForth()

If run group "checkintest", the execute order will be:

testFirst()
testThird()
 testFifth()

My question is, is there any way to implement this scenario now?

[dianny123]

Hi Any update? Could I know if this feature could be developed? I‘m appreciate if you can reply.

[krmahadevan]

@dianny123 - No there are no updates on this issue yet. Its not clear on what is the exact requirement. My thought was that, if there was an interceptor like mechanism that TestNG provides for configuration methods, then you could handle the method ordering at your side itself. But before that, I would still like to think on how we can solve this today without any changes in TestNG.

Currently a bit swamped with stuff. That is why there's a lag in the response from my side.

[krmahadevan]

@dianny123 - Here's how you can do this in your own project without needing any changes in TestNG.

1. Define a custom annotation that is equivalent to @BeforeMethod and which has priorities.
2. Build a custom listener that is capable of filtering methods that have the custom annotation.
3. Have one @BeforeMethod annotated method, that belongs to all the groups, and which filters the actual execution.

Here's a sample that shows all of this in action.

Custom annotation looks like below:

import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;

@Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
@Target(java.lang.annotation.ElementType.METHOD)
@Documented
public @interface OrderedBeforeMethod {
  String[] groups() default {};

  int priority() default 0;
}

Here's how the listener looks like :

import java.lang.reflect.Method;
import java.util.Comparator;
import java.util.List;
import java.util.stream.Collectors;
import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;
import org.testng.internal.ClassHelper;

public class OrderingListener implements IInvokedMethodListener {
  static final String KEY = "configuration";

  @Override
  public void beforeInvocation(IInvokedMethod im, ITestResult tr) {
    if (!tr.getMethod().isBeforeMethodConfiguration()) {
      return;
    }
    List<String> groups = tr.getTestContext().getCurrentXmlTest().getIncludedGroups();
    Class<?> clazz = tr.getInstance().getClass();
    List<Method> methods =
        ClassHelper.getAvailableMethods(clazz).stream()
            .filter(method -> filter(method, groups))
            .sorted(Comparator.comparingInt(this::priority))
            .collect(Collectors.toList());
    tr.setAttribute(KEY, methods);
  }

  @Override
  public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
    //Remove reference to the custom attribute and facilitate garbage collection
    testResult.setAttribute(KEY, null);
  }

  private int priority(Method method) {
    return method.getAnnotation(OrderedBeforeMethod.class).priority();
  }

  private boolean filter(Method method, List<String> includedGroups) {
    OrderedBeforeMethod bm = method.getAnnotation(OrderedBeforeMethod.class);
    if (bm == null) {
      return false;
    }
    String[] actualGroups = bm.groups();
    for (String eachGroup : actualGroups) {
      if (includedGroups.contains(eachGroup)) {
        return true;
      }
    }
    return false;
  }
}

Here's how the test class looks like:

import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.List;
import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(OrderingListener.class)
public class SampleTestCase {
  @OrderedBeforeMethod(
      groups = {"functest", "checkintest"},
      priority = 0)
  public void testFirst() {
    Reporter.log("testFirst()", true);
  }

  @OrderedBeforeMethod(
      groups = {"functest", "performance"},
      priority = 1)
  public void testSecond() {
    Reporter.log("testSecond()", true);
  }

  @OrderedBeforeMethod(
      groups = {"checkintest", "performance"},
      priority = 2)
  public void testThird() {
    Reporter.log("testThird()", true);
  }

  @Test(groups = {"functest", "checkintest"})
  public void testMethod() {
    Reporter.log("Test Method", true);
  }

  @BeforeMethod(groups = {"functest", "checkintest"})
  @SuppressWarnings("unchecked")
  public void runConfiguration() throws InvocationTargetException, IllegalAccessException {
    ITestResult result = Reporter.getCurrentTestResult();
    List<Method> methods = (List<Method>) result.getAttribute(OrderingListener.KEY);
    if (methods == null) {
      return;
    }
    for (Method method : methods) {
      method.invoke(this);
    }
  }
}

And here's how the suite xml file looks like

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="2227_suite" parallel="false" configfailurepolicy="continue">
  <test name="2227_test" verbose="2">
    <groups>
      <run>
        <include name="checkintest"/>
      </run>
    </groups>
    <classes>
      <class name="com.rationaleemotions.github.issue2227.SampleTestCase"/>
    </classes>
  </test>
</suite>

Let me know if this helps ?

[dianny123]

@krmahadevan Thank you for your reply. The solution is helpful to me.

[krmahadevan]

@dianny123 - Do you feel that this issue still needs to be kept open or do you think that we can close this off with this solution that I suggested ?

[dianny123]

yes, this issue can be closed. Thank you!