tox does not test against 2to3-converted code

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create a project written in python 2.x -- I'm using PyHamcrest as a testbed, 
here
2. add use_2to3=True to the setup args
3. run tox -e py31

What is the expected output? What do you see instead?

It should run the tests using the python3 converted code. What I see instead is 
clear evidence -- by way of syntax errors that 2to3 can fix -- that it's 
running the Python 2 code in the application directory.

What version of the product are you using? On what operating system?

$ tox --version
0.9 imported from 
/usr/local/Cellar/python/2.7/lib/python2.7/site-packages/tox/__init__.pyc

I can get it to use the actual tests if I changedir into the egg directory, but 
I have to guess at that:

changedir = {envdir}/lib/python3.1/site-packages/PyHamcrest-1.4-py3.1.egg

This is ugly, not least of which because the name of the egg is subject to 
change. However, this appears to be the right way to run the tests.

Original issue reported on code.google.com by off...@offby1.net on 17 Feb 2011 at 9:12

[GoogleCodeExporter]

To elaborate, there are a few other workarounds.  They each have some problems.

One is that you can make a custom 2to3 builder that puts compiled test files 
into build/tests/.  Then tox.ini or your test runner can specify that path to 
run tests in.  This is how Nose does it: 
https://bitbucket.org/jpellerin/nose/src/734bc7bc40ab/selftest.py#cl-35
The drawback is you need to use a custom 2to3 builder.

Another way is to run all your tests via import.  For example, run your tests 
as:
test_runnner mymodule.tests.py3k_suite
This is how Fudge does it: 
https://bitbucket.org/kumar303/fudge/src/9cafce359a21/tox.ini#cl-26
Pretty lame and error prone.

A more direct way is to use a custom test runner that gets the build test path 
at runtime:

import os
import PyHamcrest.tests
testdir = os.path.dirname(PyHamcrest.tests.__file__)
# eg. /prefix/lib/python3.1/site-packages/PyHamcrest-1.4-py3.1.egg/tests

run_tests(testdir)

Original comment by kumar.mcmillan on 17 Feb 2011 at 9:27

[GoogleCodeExporter]

do i assume right that both of you have projects that have tests inlined?  

For projects that have tests living within the application package the tests 
usually get converted during setup/2to3, i guess.  Maybe Tox could then help 
with converting command line arguments (i.e. test locations) in some way. 

For projects having tests outside the application package we probably need some 
support for a "copy-tests" step which would copy the tests to some per-env 
directory and could optionally invoke some (2to3) conversion script.

If anybody has any more thoughts or concrete considerations let me know.  I 
will not get to implement such a feature myself soon i think.

Original comment by holger.k...@gmail.com on 1 Mar 2011 at 4:36

• Changed state: Accepted

[GoogleCodeExporter]

I've used both packaged tests and non packaged tests.  Both approaches seem to 
have problems.  Is there really not a better way?  I feel like I'm missing it 
somehow.  Does everyone just develop single source modules without the need for 
2to3?  I know Michael's mock is single source for 2.4 - 3.x

But, yeah, I guess one solution would be to provide access to a build dir or 
deployment dir as a tox.ini variable.  Then the tests could cd into that and 
run the discovery script.

Original comment by kumar.mcmillan on 1 Mar 2011 at 5:49