use abstract syntax trees to discover tests instead of importing modules

[GoogleCodeExporter]

Several default configurations of frameworks that support nose (confirmed
for Pylons, maybe Django too?) are not fully import safe and thus need
custom directory configurations for nose to run an app's test suite
correctly.  See this thread for further discussion:
http://groups.google.com/group/nose-dev/browse_thread/thread/fec291197853385f

The problem can be avoided if the discovery of nose test objects is done
with abstract syntax trees (AST) instead of importing and introspecting.

As for an implementation, the stdlib module "compiler" provides AST hooks:
http://python.org/doc/2.2.3/lib/compiler.html ... and the pydoctor source
code has an example of something similar using compiler:
http://codespeak.net/svn/user/mwh/pydoctor/trunk/pydoctor/astbuilder.py

One approach could be to try building this as a plugin for a proof of
concept, however it would benefit most as a core feature, IMHO.  

If I get some time I'll start a work branch for this and put a comment
here.  If anyone else has time to get this started sooner, by all means,
hack away and just post a comment here for where the work branch is.

Original issue reported on code.google.com by kumar.mcmillan on 16 Oct 2007 at 7:42

[GoogleCodeExporter]

This should close out #110 (import problems when discovering doctests).

Also, it's probably more worthwhile to forego implementing this as a plugin and 
just
try it directly in core.

Original comment by kumar.mcmillan on 16 Oct 2007 at 9:16

[GoogleCodeExporter]

FYI : created work branch ast_discovery but haven't committed anything yet.  
Just
poking around.

Original comment by kumar.mcmillan on 17 Oct 2007 at 4:08

[GoogleCodeExporter]

some discussion about progress is here: 
http://groups.google.com/group/nose-dev/browse_thread/thread/c609106ecda1241a

Original comment by kumar.mcmillan on 19 Oct 2007 at 4:31

[GoogleCodeExporter]

hopefully I'll be able to pick up on this soon, but just for record keeping: a
rudimentary proof of concept is checked in to the ast_discovery branch.  It 
shows how
test names can be extracted from python files without importing them.

Original comment by kumar.mcmillan on 20 Nov 2007 at 4:49

[GoogleCodeExporter]

A bit ago I submitted a proposal to give a talk at PyCon on using abstract 
syntax
trees for introspection, focusing on the compiler module.  However, one reviewer
noted that compiler has been removed in python 3, hehe.  The favorable way will 
be to
use the AST parser developed for 2to3 (
http://svn.python.org/view/sandbox/trunk/2to3/ ), which uses the tokenize module
IIRC.  The same reviewer noted that this is not ready yet though because no one 
"has
done the work to implement an API for compiling bytecode from an AST passed in 
from
userspace."

Anyway, it makes me wonder if work should continue with compiler.ast or not.  
It is
currently only using limited visit hooks to get module entities so the code that
would need changing will probably be minimal.

Original comment by kumar.mcmillan on 19 Jan 2008 at 12:42

[GoogleCodeExporter]

The new _ast module now exists in Python trunk.  This will be available in 2.6 
and
3.x : http://pyside.blogspot.com/2008/03/ast-compilation-from-python.html
(announcement) and http://docs.python.org/dev/library/_ast (docs)

Also note that ast discovery solves the executable problem -- that is, how nose 
skips
files that are executable for fear of them not being import safe.  This is a big
assumption to make and can cause many a headache where tests are skipped for no
apparent reason.

Original comment by kumar.mcmillan on 2 May 2008 at 6:39

[GoogleCodeExporter]

I vote against this, since dynamic adding of tests will not work (see
http://code.google.com/p/python-nose/issues/detail?id=188)

Original comment by miki.teb...@gmail.com on 22 May 2008 at 11:23

[GoogleCodeExporter]

Yes, that will definitely be a side effect.  Tests that are imported from other 
modules into the current one won't 
work either.  I'm not sure if this is a deal breaker or not, however.  If you 
have a test suite where the code (not 
the tests) suffers heavily from not being import safe than your nose 
functionality will be very crippled.  You 
*could* set --where path/to/tests/only/ but it will make selecting tests 
complicated.  If there was an abstract 
syntax tree discovery you would at least have an opportunity to run nose, 
albeit without test generators.  Dunno, 
still needs more thought about all the pros and cons.

Original comment by kumar.mcmillan on 23 May 2008 at 12:54

[GoogleCodeExporter]

I can confirm that skipping over executable files can cause one a headache. 
Took a bit to figure out why a 
number of my test modules where getting skipped over for no apparent reason. 
Perhaps worthy of its own 
ticket?

Original comment by eik...@gmail.com on 2 Apr 2009 at 4:44