Shared data provider between multiple test methods

[GoogleCodeExporter]

This issue is being filed as a follow-up to a discussion on the
testng-users OpenSymphony forum:

http://forums.opensymphony.com/message.jspa?messageID=318098

I have a data-driven test class with several test methods, all of which
share a common data source. I would like to iterate across a large number
of data objects -- too many to instantiate and fit in memory at once. The
Iterator approach for lazy data providers seems like a good solution,
except that it does not execute the test methods in an order that would
allow for "finished" data objects to be garbage collected.

That is, I want to run methods in this order:

[instantiate bigData1]
testA(bigData1)
testB(bigData1)
[trash bigData1, instantiate bigData2]
testA(bigData2)
testB(bigData2)
etc.

Instead, what happens is this:
[instantiate bigData1]
testA(bigData1)
[instantiate bigData2]
testA(bigData2)
testB(bigData1)
testB(bigData2)

To illustrate, I have attached three toy test suite implementations:

SharedDataProviderTest1 uses the lazy Iterator approach. It instantiates
one Iterator each time the method is called, but the Iterators point into a
common (static) data structure.

SharedDataProviderTest2 uses a test factory approach instead, which results
in the methods being executed in the correct order. However, this order is
probably an implementation detail of TestNG, and not actually guaranteed.
In addition, no method dependencies can be used, or things stop working.

SharedDataProviderTest3 uses the same test factory approach, but introduces
an "initialTest" method dependency, to demonstrate how things break in this
situation. (It actually looks like there might be a bug in the dependency
resolution in this situation. Rather than explain further, it is probably
easier to just run SharedDataProviderTest3 and see for yourself.)

I think what I need is a new mechanic within TestNG to flag methods as
actually sharing a common data source, rather than just calling the same
data provider method (which as far as TestNG knows may or may not return
the exact same data with each method call).

I am using TestNG 5.7 on Mac OS X and Linux.

Original issue reported on code.google.com by ctrueden...@gmail.com on 27 Feb 2008 at 8:33

Attachments:

• SharedDataProviderTest1.java
• SharedDataProviderTest2.java
• SharedDataProviderTest3.java

[GoogleCodeExporter]

Darn, I was googling all over to find out how to do this in TestNG.  Looks like 
the
answer is "you can't, not without hacking it together, anyway".

To reiterate, I want one row of data from the dataprovider to be passed to 
several
tests before moving on to the next row of data.

Original comment by jeffrey.m.weiss@gmail.com on 21 Jan 2009 at 8:50

[GoogleCodeExporter]

Original comment by nalin.ma...@gmail.com on 20 Jun 2010 at 2:32

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

I want something similar to this, but not quite the same.  Now that there are 
parallel dataProviders and dynamic run order, what I want is not many steps 
away.  

If method A and method B both use the same dataProvider, and method B depends 
on method A, I want (optional) finer granularity in that dependency.  

Right now:  First method A works through all of the items in the dataProvider, 
then method B starts pulling from the dataProvider.  

What I want:  As method A finishes processing items in the dataProvider, those 
items become available for method B to process.  Methods A and B can optionally 
run in parallel with themselves.  In a way, this still fulfills the dependency 
promise, because for any given item, method A is executed & finished before 
method B starts.  All of this would require dynamic analysis of which items are 
ready at any given point, since there are no guarantees about the relative 
speeds of items going through method A.  

Original comment by OrigamiM...@gmail.com on 3 Jul 2010 at 10:35

[GoogleCodeExporter]

I met the same problem. I am not able to solve it also. I added dataProvider 
annotation to Class level but then you don't have any options to get data 
inside methods.

Original comment by roman.ra...@gmail.com on 31 Aug 2012 at 2:00

[GoogleCodeExporter]

I am facing the similar issue and need to use the shared dataprovider across 
different tests as stated in the original post above. Please let me know if any 
of you found a solution to this ?

Original comment by shikss.a...@gmail.com on 28 Nov 2012 at 7:49

[GoogleCodeExporter]

> Please let me know if any of you found a solution to this

My project (Bio-Formats) works around the issue using the test factory approach 
illustrated in SharedDataProviderTest2 above.

It is vital that each test class instance not actually initialize its big data 
structures until its "before-class" annotated method is called (rather than 
doing so in the constructor). And then when the "after-class" annotated method 
is called, the big data structures must be cleared.

As long as you don't have any method dependencies, TestNG calls all test 
methods of a particular instance before moving on to the next instance, as 
illustrated in SharedDataProviderTest2.

Relevant Bio-Formats classes illustrating the technique:
  * https://github.com/openmicroscopy/bioformats/tree/master/components/test-suite/src/loci/tests/testng/FormatReaderTest.java
  * https://github.com/openmicroscopy/bioformats/tree/master/components/test-suite/src/loci/tests/testng/FormatReaderTestFactory.java

This approach does limit you to 2^31-1 test class instances, which has been 
more than sufficient for my use cases.

As noted above, the execution order may be an implementation detail of TestNG, 
but in practice it works.

Original comment by ctrueden...@gmail.com on 28 Nov 2012 at 6:29

[GoogleCodeExporter]

Is there a workaround?

I need to be able to run multiple tests sharing the same data provider...

Original comment by Nivea...@gmail.com on 6 Mar 2014 at 8:27

[GoogleCodeExporter]

Plus, my tests have to run in a particular order...

Original comment by Nivea...@gmail.com on 6 Mar 2014 at 8:29