A test suite for GC3Pie

[GoogleCodeExporter]

We badly need an automated test suite for GC3Pie.

This is difficult, as many functionalities depend on interacting with
external entities, like the grid information system or a remote
cluster.  On the other hand, properly modelling these interactions
through a mock library can help us better understanding the possible
failures and should, in general lead to cleaner code.

This is a sort of "index" ticket that depends on the actual steps
taken to implement a roadmap to having an automated test suite for
GC3Pie. 

My roadmap proposal:

a. Start using an automated testing system, e.g., TOX[1] and have it
   run the few doctests that we already have (in the `gc3libs.units`
   and `gc3libs.url` packages, for example)

b. Use Pythoscope[2] to generate an initial skeletal test suite.

c. Decide on a mock object framework/library[3] and use it to model
   interactions with the external entities.  Deciding might involve
   writing the same test case with a few candidates mock objects
   libraries and assessing what is more convenient.

d. Use a command-line testing framework[4],[5] to also check that the
   command-line scripts actually print what they are supposed to print
   and do that in the right sequence.  A package like this can help.

Any comments?  Do you see any obvious flaws and/or missing points?

[1]: http://pypi.python.org/pypi/tox
[2]: http://pythoscope.org/
[3]: http://www.voidspace.org.uk/python/mock/compare.html
[4]: http://pythonpaste.org/scripttest/
[5]: http://pypi.python.org/pypi/cram

Original issue reported on code.google.com by riccardo.murri@gmail.com on 20 Feb 2012 at 1:49

• Blocked on: #243

[GoogleCodeExporter]

I think that point a. is *very high-priority*, as we will not start writing and 
using tests until we have an automated test suite.

Original comment by riccardo.murri@gmail.com on 20 Feb 2012 at 1:50

[GoogleCodeExporter]

Original comment by riccardo.murri@gmail.com on 20 Feb 2012 at 1:56

[GoogleCodeExporter]

Original comment by riccardo.murri@gmail.com on 20 Feb 2012 at 1:56

• Changed title: A test suite for GC3Pie

[GoogleCodeExporter]

A comprehensive list of Python testing tools:

http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy

Original comment by riccardo.murri@gmail.com on 22 Feb 2012 at 3:39

[GoogleCodeExporter]

Today I've tested nose[1] and pytest. There are debian packages for
them. At first glance they look quite similar, even though py.test
seems to do a bit more magic than I would appreciate...

I haven't tested tox yet (will do soon), but they say it works well
with both pytest and nosetests, so I would consider the following
options for now:

* tox
* tox + nose
* tox + pytest

I personally don't like the pythoscope approach: I'd rather create
tests for what I actually want to test than test *any single method*
of my classes.

[1] http://readthedocs.org/docs/nose/en/latest/index.html

Original comment by arcimbo...@gmail.com on 22 Feb 2012 at 9:34

[GoogleCodeExporter]

| I personally don't like the pythoscope approach: I'd rather create
| tests for what I actually want to test than test *any single method*
| of my classes.

For clarification: I think Pythoscope might be a way to bootstrap
the test code; after that, we need to write and maintain the tests on
our own...

Original comment by riccardo.murri@gmail.com on 23 Feb 2012 at 10:58