mahmoudimus / nose-timer

A timer plugin for nosetests (how much time does every test take?)
MIT License
126 stars 33 forks source link

Add pypy, pypy3, 3.2 but allow pypy3 and 3.2 to fail #50

Closed hugovk closed 9 years ago

hugovk commented 9 years ago

Some Python environments are missing from the CI, so add them:

However, there's a problem with pypy3 and 3.2 at the moment (https://github.com/mahmoudimus/nose-timer/issues/51) so allow them to fail until a fix is found.

skudriashev commented 9 years ago

@hugovk, Python 3.2 was excluded some time ago, because multiprocessing.Manager().dict() doesn't work properly on that version. I don't think this i s good idea to add something that doesn't work. I would add only those envs that works fine right now and that are described in setup.py.

hugovk commented 9 years ago

(The `allowed_failures thing didn't work for some reason...)

Fair enough about excluding 3.2. But this means other projects that do need to support 3.2 and PyPy3 cannot use nose-timer.

It would require an ugly workaround something like this (hacky pseudocode, not tested):

- if [ ${TRAVIS_PYTHON_VERSION} == "3.2" ]; then
   coverage run --append --include=PIL/* -m nose -vx Tests/test_*.py; else
    if [ ${TRAVIS_PYTHON_VERSION} == "pypy3" ]; then
     coverage run --append --include=PIL/* -m nose -vx Tests/test_*.py; else
         coverage run --append --include=PIL/* -m nose -vx Tests/test_*.py --with-timer --timer-top-n 10 --timer-ok 1s --timer-warning 10s
    fi
  fi

So rather than require special handling to avoid timing out the tests altogether, could nose-timer instead let the tests run without timing?

That way a single test line could be used for all versions:

coverage run --append --include=PIL/* -m nose -vx Tests/test_*.py --with-timer --timer-top-n 10 --timer-ok 1s --timer-warning 10s

Then all tests run and we get timings for 2.7, 3.4 etc but not for 3.2 etc.

Or do you suggest another way to use nose-timer for projects that need to support all these Python version?

Thanks!

skudriashev commented 9 years ago

@hugovk, still I can't understand what is the point of testing it if it still doesn't work for end user?

hugovk commented 9 years ago

@skudriashev The point is that nose will still run all the unit tests for all Python versions, and for those that support timings, we get those timings on those.

Our unit tests are running on all the Python versions supported by Travis: 2.6, 2.7, 3.2, 3.3, 3.4. PyPy and PyPy3. It'd be nice to add timings for those that support have timings.

skudriashev commented 9 years ago

@hugovk, but we doesn't claim Python 3.2 is supported, right? In the worst case we should add it to setup.py, shouldn't we?

BTW, adding Python 3.2 here means nose-tests should be fixed later?

skudriashev commented 9 years ago

@mahmoudimus, @e0ne, what do you think about this?

mahmoudimus commented 9 years ago

@hugovk @skudriashev if we have to support Python 3.2 because this is a testing module, then we should maybe schedule a way for nose-timer to work with Python 3.2 (even if multiprocessing.Manager().dict() doesn't work, though I guess it's about time and effort).

skudriashev commented 9 years ago

@hugovk, any updates on this?

hugovk commented 9 years ago

@skudriashev This is working after merging with your latest changes to master :)

I've removed the allowed_failures bit.

https://travis-ci.org/hugovk/nose-timer/builds/43334098

skudriashev commented 9 years ago

@hugovk, I've just fixed Python3.2 and PyPy3. Closing this one.

hugovk commented 9 years ago

:+1: