Add a listener for data provider interception

[juherr]

From #1073

https://groups.google.com/forum/#!topic/testng-dev/WASvXfvq3_M

[krmahadevan]

@juherr - Would it be possible to call out explicitly what we are trying to add, as part of this feature request ? Do we need a

• beforeDataProviderExecution() and
• afterDataProviderExecution()

or are we after something else ?

[juherr]

@baev Is before/after data provider will be enough for you?

[baev]

Yep. Also there is should be a way to get all the tests that use such data provider.

[krmahadevan]

@baev

Also there is should be a way to get all the tests that use such data provider.

That might not be possible, because TestNG doesn't invoke a data provider till it gets to the point of invoking a test. So I am not sure as to how does one provide the cross reference of data provider -> list of methods that use it.

[baev]

What about test -> dataprovider link?

[krmahadevan]

I currently have worked on building a listener which looks like this

public interface IDataProviderListener extends ITestNGListener {

    /**
     * This method gets invoked just before a data provider is invoked.
     *
     * @param method - The {@link ITestNGMethod} method that is going to consume the data
     *               provided by the data provider.
     */
    void beforeDataProviderExecution(ITestNGMethod method);

    /**
     * This method gets invoked just after a data provider is invoked.
     *
     * @param method - The {@link ITestNGMethod} method that received the data
     *               provided by the data provider.
     */
    void afterDataProviderExecution(ITestNGMethod method);
}

This would basically provide you with the @Test method that is supposed to receive data from a particular data provider.

[baev]

Few questions:

1. How to get information about data provider method? I mean its method name, class name, annotations declared on this method.
2. Can data provider be used by few tests? What is the behavior in that case?

[krmahadevan]

@baev - I have changed the implementation to be something like below

package org.testng;

/**
 * A listener that gets invoked before and after a data provider is invoked by TestNG.
 */
public interface IDataProviderListener extends ITestNGListener {

    /**
     * This method gets invoked just before a data provider is invoked.
     *
     * @param method                  - The {@link ITestNGMethod} method that is going to consume the data
     *                                provided by the data provider.
     * @param dataProviderInformation - A {@link DataProviderInformation} object that contains details about the
     *                                data provider that is about to be executed.
     */
    void beforeDataProviderExecution(ITestNGMethod method, DataProviderInformation dataProviderInformation);

    /**
     * This method gets invoked just after a data provider is invoked.
     *  @param method             - The {@link ITestNGMethod} method that received the data
     *                           provided by the data provider.
     * @param dataProviderInformation - A {@link DataProviderInformation} object that contains details about the
     */
    void afterDataProviderExecution(ITestNGMethod method, DataProviderInformation dataProviderInformation);
}

And the DataProviderInformation POJO looks like below :

import org.testng.annotations.IDataProviderAnnotation;

import java.lang.reflect.Method;
import java.util.Collections;
import java.util.List;

/**
 * Represents information pertaining to a data provider that would be invoked.
 */
public class DataProviderInformation {
    private Object instance;
    private Method method;
    private String name;
    private boolean parallel;
    private List<Integer> indices;

    public DataProviderInformation(IDataProviderAnnotation annotation, Method method, Object instance) {
        this.name = annotation.getName();
        this.parallel = annotation.isParallel();
        this.indices = Collections.unmodifiableList(annotation.getIndices());
        this.method = method;
        this.instance = instance;
    }

    public Object getInstance() {
        return instance;
    }

    /**
     * @return - A {@link Method} object that represents the actual
     * {@literal @}{@link org.testng.annotations.DataProvider} method.
     */
    public Method getMethod() {
        return method;
    }

    /**
     * @return - The name of the data provider.
     */
    public String getDataProviderName() {
        return name;
    }

    /**
     * @return - <code>true</code> if the data provider would run in parallel.
     */
    public boolean isParallel() {
        return parallel;
    }

    /**
     * @return - A List of {@link Integer} that represents Which indices to run from this data provider.
     */
    public List<Integer> getIndices() {
        return indices;
    }
}

[krmahadevan]

@baev - That should basically tell you details about the @DataProvider method as well in the listener.

[baev]

LGTM in general.

BTW Is it worth to introduce DataProviderInformation POJO? Can we use ITestNGMethod instead?

[baev]

feel free to summon me to review the PR with this changes

[krmahadevan]

BTW Is it worth to introduce DataProviderInformation POJO? Can we use ITestNGMethod instead?

@baev - I don't think it makes sense to view a @DataProvider annotated method as an ITestNGMethod because ITestNGMethod exposes a lot of things which just aren't applicable for a @DataProvider. For e.g., org.testng.ITestNGMethod#getRetryAnalyzer (There is no retry analyzer associated with a @DataProvider). So that is not going to be part of the changeset. It will contain the POJO

[cbeust]

1. You can find out which methods invoke which data providers simply by introspecting the classes and looking for @Test and @DataProvider.

    private Object instance;
    private Method method;
    private String name;
    private boolean parallel;
    private List<Integer> indices;

We shouldn't duplicate the information that can be retrieved from the @DataProvider attributes. If the user wants those, they can just look up the @DataProvider annotation and read those themselves.

[krmahadevan]

@cbeust - I am not sure if one can figure out the instance to which a @DataProvider method belongs to using reflection. So it makes sense for us to share the instance as well.

[cbeust]

Yes, only remove whatever can be found on the attributes of @DataProvider so we don't introduce duplication (obviously, instance is not one of them).

[krmahadevan]

@baev - FYI: The PR for this is available https://github.com/cbeust/testng/pull/1505 I am not able to ping you from the PR. So am pinging you from here. Please take a look

[AdityaDogra]

Can you guys help me to figure out how to listen to a specific failure in one of the runs when dataprovider is used. As we have ITestListner do have any specific listener for dataprovider also.How dataprovider keeps track of the failures using TestNG.