use abstract syntax trees to discover tests instead of importing modules

[jpellerin]

Several default configurations of frameworks that support nose (confirmed for Pylons, maybe Django too?) are not fully import safe and thus need custom directory configurations for nose to run an app's test suite correctly. See this thread for further discussion: http://groups.google.com/group/nose-dev/browse_thread/thread/fec291197853385f

The problem can be avoided if the discovery of nose test objects is done with abstract syntax trees (AST) instead of importing and introspecting.

As for an implementation, the stdlib module "compiler" provides AST hooks: http://python.org/doc/2.2.3/lib/compiler.html ... and the pydoctor source code has an example of something similar using compiler: http://codespeak.net/svn/user/mwh/pydoctor/trunk/pydoctor/astbuilder.py

One approach could be to try building this as a plugin for a proof of concept, however it would benefit most as a core feature, IMHO.

If I get some time I'll start a work branch for this and put a comment here. If anyone else has time to get this started sooner, by all means, hack away and just post a comment here for where the work branch is.

Google Code Info: Issue #: 116 Author: kumar.mcmillan Created On: 2007-10-16T19:42:25.000Z Closed On:

[jpellerin]

This should close out #110 (import problems when discovering doctests).

Also, it's probably more worthwhile to forego implementing this as a plugin and just try it directly in core.

Google Code Info: Author: kumar.mcmillan Created On: 2007-10-16T21:16:54.000Z

[jpellerin]

FYI : created work branch ast_discovery but haven't committed anything yet. Just poking around.

Google Code Info: Author: kumar.mcmillan Created On: 2007-10-17T16:08:44.000Z

[jpellerin]

some discussion about progress is here: http://groups.google.com/group/nose-dev/browse_thread/thread/c609106ecda1241a

Google Code Info: Author: kumar.mcmillan Created On: 2007-10-19T16:31:10.000Z

[jpellerin]

hopefully I'll be able to pick up on this soon, but just for record keeping: a rudimentary proof of concept is checked in to the ast_discovery branch. It shows how test names can be extracted from python files without importing them.

Google Code Info: Author: kumar.mcmillan Created On: 2007-11-20T16:49:33.000Z

[jpellerin]

A bit ago I submitted a proposal to give a talk at PyCon on using abstract syntax trees for introspection, focusing on the compiler module. However, one reviewer noted that compiler has been removed in python 3, hehe. The favorable way will be to use the AST parser developed for 2to3 ( http://svn.python.org/view/sandbox/trunk/2to3/ ), which uses the tokenize module IIRC. The same reviewer noted that this is not ready yet though because no one "has done the work to implement an API for compiling bytecode from an AST passed in from userspace."

Anyway, it makes me wonder if work should continue with compiler.ast or not. It is currently only using limited visit hooks to get module entities so the code that would need changing will probably be minimal.

Google Code Info: Author: kumar.mcmillan Created On: 2008-01-19T00:42:45.000Z

[jpellerin]

The new _ast module now exists in Python trunk. This will be available in 2.6 and 3.x : http://pyside.blogspot.com/2008/03/ast-compilation-from-python.html (announcement) and http://docs.python.org/dev/library/_ast (docs)

Also note that ast discovery solves the executable problem -- that is, how nose skips files that are executable for fear of them not being import safe. This is a big assumption to make and can cause many a headache where tests are skipped for no apparent reason.

Google Code Info: Author: kumar.mcmillan Created On: 2008-05-02T18:39:29.000Z

[jpellerin]

I vote against this, since dynamic adding of tests will not work (see http://code.google.com/p/python-nose/issues/detail?id=188)

Google Code Info: Author: miki.teb...@gmail.com Created On: 2008-05-22T23:23:54.000Z

[jpellerin]

Yes, that will definitely be a side effect. Tests that are imported from other modules into the current one won't work either. I'm not sure if this is a deal breaker or not, however. If you have a test suite where the code (not the tests) suffers heavily from not being import safe than your nose functionality will be very crippled. You could set --where path/to/tests/only/ but it will make selecting tests complicated. If there was an abstract syntax tree discovery you would at least have an opportunity to run nose, albeit without test generators. Dunno, still needs more thought about all the pros and cons.

Google Code Info: Author: kumar.mcmillan Created On: 2008-05-23T00:54:05.000Z

[jpellerin]

I can confirm that skipping over executable files can cause one a headache. Took a bit to figure out why a number of my test modules where getting skipped over for no apparent reason. Perhaps worthy of its own ticket?

Google Code Info: Author: eik...@gmail.com Created On: 2009-04-02T04:44:28.000Z