Closed GoogleCodeExporter closed 8 years ago
It should be *mostly* possible with a plugin, since you could either use
prepareTest() to replace the top-level test callable with a function that forks
and
collects, or (maybe easier) use prepareTestCase() to replace each individual
test
case with a fork/collect -- or more realistically with something to place the
test
case into some kind of shared queue for one of N pre-forked runners to pick up.
Why only mostly possible: the difficulty you'll run into is handling package or
module-level fixtures when running tests in parallel. I don't see an easy
solution
for that.
But I think this is a really good idea nonetheless, so I'll keep it on the
books for
a release after 0.10. Anyone who wants to give it a try, grab this ticket.
Original comment by jpelle...@gmail.com
on 11 Sep 2007 at 2:46
There was some discussion building a plugin to run tests in parallel, across
multiple
OS processes and remote machines:
http://groups.google.com/group/nose-users/browse_thread/thread/bc0edcf6a3fc167b/
In summary:
- there should be enough hooks in the nose 0.10 plugin API for this
- making use of 3rd party "parallel" processing modules?
- the parallel python / pp module (http://www.parallelpython.com/) is probably
*not* a good fit
- the processing module (http://cheeseshop.python.org/pypi/processing) is a possibility
- Pyro (http://pyro.sourceforge.net/) is also a possibility
- testoob (http://testoob.sourceforge.net/) supports testing in multiple threads
using twisted, so this code may provide some useful insight
- twisted ;) (http://twistedmatrix.com/trac/)
In addition to Jason's comment above, I think another hurdle will be how a
"worker"
process will communicate test results (what was run, what passed, what failed,
traceback, etc) back to the "supervisor." Jason posted this link on another
topic:
http://mail.python.org/pipermail/python-3000/2007-April/006604.html which may
have
some usable code for generating picklable tracebacks.
Original comment by kumar.mcmillan
on 3 Dec 2007 at 7:25
IPython should also be considered as 3rd party option:
http://ipython.scipy.org/moin/Parallel_Computing
Original comment by kumar.mcmillan
on 4 Dec 2007 at 12:00
There is also a thread now on Testing In Python, kicking around the idea of
creating
a Pycon 2008 sprint for this project:
http://lists.idyll.org/pipermail/testing-in-python/2007-December/thread.html#463
Original comment by kumar.mcmillan
on 7 Dec 2007 at 7:08
Here are some insightful updates that were made to the Testing In Python
thread.
Both py.test and twisted's trial test runner have experimental features for
running
distributed tests in parallel :
- py.test: https://codespeak.net/py/dist/test.html#automated-distributed-testing
- trial: http://twistedmatrix.com/trac/ticket/1784
Original comment by kumar.mcmillan
on 10 Dec 2007 at 10:43
Here is a publisher/subscriber implementation (currently in beta) that may
provide an
out-of-the-box way to publish test results from remote test runners back to a
central
"test result reporter" :
http://pypubsub.sourceforge.net/
Original comment by kumar.mcmillan
on 20 Dec 2007 at 7:16
will need to create pickle-able tracebacks to send from a worker back to the
supervisor. This seems to be easy enough, swiping from John J. Lee's code in
the
nosepipe plugin, which was based on an example from Collin Winter:
http://mail.python.org/pipermail/python-3000/2007-April/006604.html
def _exc_info_to_string(self, err, test):
exctype, value, tb = err
# Skip test runner traceback levels
while tb and self._is_relevant_tb_level(tb):
tb = tb.tb_next
if exctype is test.failureException:
# Skip assert*() traceback levels
length = self._count_relevant_tb_levels(tb)
return ''.join(traceback.format_exception(exctype, value, tb,
length))
return ''.join(traceback.format_exception(exctype, value, tb))
def _is_relevant_tb_level(self, tb):
return tb.tb_frame.f_globals.has_key('__unittest')
def _count_relevant_tb_levels(self, tb):
length = 0
while tb and not self._is_relevant_tb_level(tb):
length += 1
tb = tb.tb_next
return length
Original comment by kumar.mcmillan
on 4 Jan 2008 at 5:27
I've been thinking more about the setup/teardown problem. This approach might
address it well enough:
a supervisor loops through a suite and assigns an ID number to each test. It
then
divides that by the number of workers and sends each worker a "slice," a range
of
numbers.
A more concise example: say that a supervisor cataloged a test suite and has 4
workers and 400 tests. It would execute 4 runs of the nosetests command,
something like:
nosetests 1:100
nosetests 101:200
nosetests 201:300
nosetests 301:400
if nose's collector was invoked as usual, then any setup/teardowns encountered
while
collecting those slices will run.
Original comment by kumar.mcmillan
on 4 Jan 2008 at 5:43
Tentatively scheduled for 0.11.
The slice approach won't work, unfortunately, since we can't know whether a
given
context fixture is re-entrant. I think it will be necessary instead to divide
tests
up by minimal fixture context (test, class, module or package) and dispatch the
entire context suite when there are fixtures at that level. I believe this will
require a depth-first collection run, then a 2nd pass to look at the collected
suites
and see where they can be split.
Original comment by jpelle...@gmail.com
on 6 Jun 2008 at 6:24
ah, right. However, one approach might be to lazily deploy setup/teardown to
each
slice, as needed. That is:
tests/
run setup()...
test_one.py
test_two.py
test_three.py
test_four.py
run teardown()...
then deploy slices like so, deploying fixtures where necessary:
slice 1:
run setup()...
test_one.py
run teardown()...
slice 2:
run setup()...
test_two.py
test_three.py
run teardown()...
slice 3:
run setup()...
test_four.py
run teardown()...
As depth increased, package setup/teardown would have to be stacked and the
whole
stack deployed to each slice but that could be costly (timewise) if many slices
were
used.
Original comment by kumar.mcmillan
on 6 Jun 2008 at 7:47
Also it would be great to have simpler functionality - run each test or test
case in
a separate python interpreter one by one (I know it was discussed earlier -
http://nose.python-hosting.com/ticket/77). I know it would be slow but from my
point
of view don't care much - now I use shell script to run each test case
separately and
it's not so slow. Would love to see this built in into nose so there would be
no need
to create some wrappers. And I think it could be a first step in introducing
parallel
tests.
Kind regards,
Pawel
Original comment by 111100...@gmail.com
on 13 Oct 2008 at 7:45
In trunk, closing.
Original comment by jpelle...@gmail.com
on 26 Mar 2009 at 8:09
Original issue reported on code.google.com by
vols...@gmail.com
on 11 Sep 2007 at 2:34