search

wushuyi / passlib

Automatically exported from code.google.com/p/passlib
Other
0 stars 0 forks source link

Django tests failing under 1.6.1 #50

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Run the testsuite like so
2. nosetests -w build/lib
3.

What is the expected output? What do you see instead?
Pass, fails.

What version of the product are you using? On what operating system?
1.6.1; gentoo linux

Please provide any additional information below.

From tox.ini we have;

[testenv:py27]
deps =
    nose
    coverage
    unittest2
    py-bcrypt
    bcryptor
    django
    M2Crypto

Installed are;

nose-1.3.0, coverage-3.6, unittest2 is there,
py-bcrypt-0.4, Bcryptor-1.2.2, django latest == 1.5.5,
m2crypto-0.21.1.

django 1,5,4 appeared to fair as poorly.

to get a clean tun it requires;

        nosetests -w build/lib \
                -e test_unsalted_sha1 -e test_77_fuzz_input \
                -e test_config -e test_unsalted_md5

I can't help but notice the bugs point at the unreleased 1.6.2. Do you really 
have virt. no issues in 1.6.1? euscan tells me there's no later release than 
1.6.1.

Original issue reported on code.google.com by del...@iinet.com.au on 15 Dec 2013 at 10:29

Attachments:

GoogleCodeExporter commented 9 years ago 
I can confirm those test failures.  

It looks like Django has altered some border cases in their hasher code, which 
is causing a number of `passlib.ext.django`-related tests to fail. I'm not that 
surprised, passlib's django integration tests were designed to be extra 
sensitive to changes in Django, to keep a tight leash on passlib.ext.django's 
behavior. (If you want, you can edit tox.ini to remove "django" from 
[testenv:py27], and all the tests should pass, confirming it's just the django 
integration that's failing).

It doesn't look like any of the changes impact security, but they definitely 
need to be addressed to shore up Passlib's Django support. I think this by 
itself is enough to warrant a 1.6.2 release later this week.

And yeah, while there are a few small enhancements slated for 1.6.2, the 1.6.1 
release hasn't had any reported bugs until now :)

Btw, thank you very much for the detailed report!

- Eli

Original comment by elic@astllc.org on 16 Dec 2013 at 3:27

GoogleCodeExporter commented 9 years ago 
y'r welcome.  I think I'll await the 1.6.2 for now.  Thx for your reply

Original comment by del...@iinet.com.au on 17 Dec 2013 at 7:35

GoogleCodeExporter commented 9 years ago 
1.6.2 has been released, and all test are passing for Django 1.2 - 1.6 now (at 
least on my systems). I'm closing this issue, but please let me know if any of 
your test runs are still failing... more environments == better testing :)

- Eli

Original comment by elic@astllc.org on 27 Dec 2013 at 12:35

GoogleCodeExporter commented 9 years ago 
~/cvsPortage/gentoo-x86/dev-python/passlib $ PYTHON_TARGETS=pypy2_0 ebuild 
passlib-1.6.2.ebuild clean test

yields

Ran 1926 tests in 131.211s

FAILED (SKIP=613, errors=49)

with a plethora of

ERROR: django_bcrypt (pybcrypt backend): test encrypt() / genconfig() PLUS 
various individual test errors

gees it's pypy. what's your take on pypy? Are you inclined to requesting a full 
log of error output for running under pypy?

~/cvsPortage/gentoo-x86/dev-python/passlib $ PYTHON_TARGETS=python2_7 ebuild 
passlib-1.6.2.ebuild clean test

yields

 * python2_7: running distutils-r1_run_phase python_test
................................................................................
..................................................S......SSS....S.SSS........S..
....SSS....S.SSS........S......SSS....S.SS.S........SSSSSSSSSS....S.SS.S........
........S......SS.S.......SSSSSSSSSS....S.SSSS..........SS.SSS.SSS....S.SS.S....
.....S......SSS....S.SSSS.......S........S....S.SSSS..............SSS......SS.S.
.........S........S....S.SSSS.......S........S....S.SS..........S........S....S.
SSSS.......SSSSSSSSSS....S.SSSS.......SSSSSSSSSS....S.SSSS.......SSSSSSSSSS....S
.SSSS.......SSSSSSSSSS....S.SSSS.......SSSSSSSSSS....S.SSSS.......SSSSSSSSSS....
S.SS.S...S.............SSS......SSS......S....SSSSSSSSSS....S.SSSS........SSSSSS
SSSS....S.SS.S.......S......SSS....S.SSS........S......SSS....S.SSS........SSSSS
SSSSS....S.SSSS.......SSSSSSSSSS....S.SS.S...............SSS......SSS...........
SSSSSSSSSS....S.SSSS..........SSSSSSSSSS....S..SSS..........S......SSS....S.....
.......S......SSS....S............SSSSSSSSSS....S.SS.S........SSSSSSSSSS....S.SS
.S.......SSSSSSSSSS....S.SS.S.......SSSSSSSSSS....S.SS.S..........S......SSS....
S.SSSS.......S........S....S.SSS........S........S....S.SSSS.......S........S...
.S.SSSS.......S.............S.SSS........SSSSSSSSSS....S.SSSSS......SSSSSSSSSS..
..S.SS.S..........S........S....S.SSS.........................S......SSS........
.........S......S.S....................S......S.S...........S........S....S.S.S.
.......SSSSSSSSSS....S.SS.S..S.....SSSSSSSSSS....S.SSSSS.S......................
S......S.............S.....S.S.S.................S.....S.SSSS...SS.............S
......SSSS.........S......SSS....S..S...........SSSSSSSSSS....S..SSS..S....S....
....S....S.SSSS.........S........S....S.SSSS.........S......SSS....S.SS.........
..S......SSS....S.SS............................................................
.....................................S......SSS....S.SSSS...................SSSS
SSSSSS....S.SSSS......
----------------------------------------------------------------------
Ran 1942 tests in 116.199s

OK (SKIP=591)

with

nosetests -w "${BUILD_DIR}"/lib \
        || die "Tests fail with ${EPYTHON}"

which amounts to about PYTHONPATH=. nosetests .

Original comment by del...@iinet.com.au on 27 Dec 2013 at 8:17

GoogleCodeExporter commented 9 years ago 
I am trying to support pypy if possible, so definitely interested in the full 
error log if it's not too much trouble (if you want to put it up on 
gist.github.com that'd probably be better than pasting it here).  

There's so many library combinations, looks like one slipped through the 
cracks. I'm guessing you're running the test under pypy with django installed, 
but without bcrypt? I'm curious, if you install "bcrypt" (not "py-bcrypt") as 
well, do the tests pass then? Also, along with the log if you could tell me 
what pypy & django versions you're testing under.

Thanks!
- Eli

Original comment by elic@astllc.org on 27 Dec 2013 at 2:34

GoogleCodeExporter commented 9 years ago

Original comment by elic@astllc.org on 27 Dec 2013 at 2:35

GoogleCodeExporter commented 9 years ago 
Oh dang it!!!!!!

[I] dev-python/Bcryptor
     Available versions:  (~)1.2.2 (~)1.2.2[1] {test PYTHON_TARGETS="python2_6 python2_7"}
     Installed versions:  1.2.2[1](15:44:22 15/12/13)(-test PYTHON_TARGETS="python2_6 python2_7")
     Homepage:            http://www.bitbucket.org/ares/bcryptor/
     Description:         Python wrapper for bcrypt

[D] dev-python/django
     Available versions:  1.4.8 (~)1.5.4 **9999 {doc mysql postgres sqlite test vhosts PYTHON_TARGETS="python2_6 python2_7 python3_2 python3_3"}
     Installed versions:  1.5.5(17:03:23 15/12/13)(-doc -mysql -postgres -sqlite -test PYTHON_TARGETS="python2_6 python2_7 python3_2 python3_3")

[I] dev-python/pypy
     Available versions:  
     (2.0)  (~)2.0.2

testuser@archtester ~/cvsPortage/gentoo-x86/dev-python/passlib $ qlist Bcryptor
/usr/lib64/python2.7/site-packages/bcryptor/bcrypt.py
plus the rest.

so, I have the package Bcryptor installs a file bcryptor/bcrypt.py.

Now I shall refrain from rushing to pypi and doing a search on pypi.org for all 
packages with the string and just ask you here.  Are you referring to the 
package Bcryptor?  I think yes.  Your tox.ini has;

[testenv:py-bcrypt-py3]
basepython = python3
deps =
#    bcryptor -- has py3 installation issues
    py-bcrypt
    {[testenv:py32]deps}
commands =
    nosetests {posargs:passlib.tests.test_handlers_bcrypt}

[testenv:pypy]
# pypy (as of v1.6 - v2.2) targets Python 2.7
basepython = pypy
deps =
    bcrypt
    {[testenv]deps}

Now recently I have had not one but 2 packages cause me confusion come grief by 
way of the package name.  I specially added Bcryptor to portage because I found 
it in the tox.ini of the prior version.  Now you ask do I have bcrypt 
installed.  At this stage I simply don't know for sure.  

Now, I added Bcryptor with only py2.6 2.7 support because of this.

~/cvsPortage/gentoo-x86/dev-python/Bcryptor $ ebuild Bcryptor-1.2.2.ebuild 
clean install

yielded

warning: build_py: byte-compiling is disabled, skipping.

running build_ext
building 'bcryptor.engine' extension
error: unknown file type '.pyx' (from 'lib/bcryptor/wrap/engine.pyx')

so the source of the bcryptor or Bcryptor fooled the installed pypy on its 
attempt to build the source code.  This is likely a cython thingy.

That py3 has installation issues is consistent with my findings to have added 
the package Bcryptor without py3 support.   However it seems you have SOMETHING 
installed that props up pypy for doing the testsuite pulling on a bcrypt.py.

I think the answer lies in getting these deps clearly understood and the log of 
the output may ultimately not be required and is only an issue because it's 
quite long and too long to put here.

I do like your support of pypy, it's coming on nicely over time and really 
needs devs like you to embrace it.  I await your response.

Original comment by del...@iinet.com.au on 28 Dec 2013 at 3:43

GoogleCodeExporter commented 9 years ago 
Yeah, the package naming for the bcrypt libraries in particular is rather poor. 
 My tox files probably aren't the best place to find answers, either, they're 
rather a mess at this point :)

There's actually three main bcrypt libraries for python. I'm not sure what 
they're named under other packaging systems like portage, but on pypi they're 
named  'bcryptor' (https://pypi.python.org/pypi/Bcryptor), 'py-bcrypt' 
(https://pypi.python.org/pypi/py-bcrypt/), and 'bcrypt' 
(https://pypi.python.org/pypi/bcrypt/1.0.2). 

Personally, I'd avoid 'bcryptor'... I support it in passlib, but it's cython 
bindings always cause me grief, and also make it incompatible with pypy (as far 
as I can tell). It certainly sounds like it's managed to half-install itself 
and foul things up on your system. 

'py-bcrypt' is the oldest of the three, and it's pretty solid. But it's also a 
plain old C extension, and won't work with pypy.

'bcrypt' is the one you want. It's a cffi-based extension, so it's the only one 
of the three which will work under pypy as well as normal C python. It's also 
the "something" that was being used in my tox config to test passlib with pypy. 
I'd recommend eliminate all traces of the others, and trying to get that one 
installed. 

Let me know if that helps straighten things out!

- Eli

Original comment by elic@astllc.org on 28 Dec 2013 at 4:40

GoogleCodeExporter commented 9 years ago 
right it does straighten things out.  I suspected it to be a separate package, 
so I shall get it and make an ebuild and install it to the system to give it a 
full whirl.  Your interest and co-operation warrants it, but whether it finds 
its way to portage proper is another matter.  I'll prob. do that tomorrow.

Original comment by del...@iinet.com.au on 28 Dec 2013 at 12:47

GoogleCodeExporter commented 9 years ago 
Eli; righty.

What I have after all this further testing is a headache.  I have in fact made 
an ebuild and installed it into my system by gentoo dev know how, and yep 
pypy2_0 does in fact pass the whole testsuite.  I uninstalled the other 2, and 
then I got 3 Errors 1 Fail under py3, and 3 Errors under py2.  This makes for a 
headache.  On re-installing py-bcrypt they disappeared, however your test setup 
has finally defeated portage's capacity to cope by a 50m. pole vault.  

The tox.ini and the whole testsuite has a separate list of req'd deps which 
once installed see it pass under all python 'impls'.  In gentoo to cater to 
this we need install any and all deps, and have it covered for any possible 
scenario or python impl.  Here is the problem.

~/improvise/dev-python/bcrypt $ sudo ebuild  bcrypt-1.0.2.ebuild clean merge

===========================================================
 * dev-python/py-bcrypt-0.4:0::gentoo
 *  /usr/lib64/pypy2.0/site-packages/bcrypt/__init__.py
 *  /usr/lib64/python2.6/site-packages/bcrypt/__init__.py
 *  /usr/lib64/python2.7/site-packages/bcrypt/__init__.py
 *  /usr/lib64/python3.2/site-packages/bcrypt/__init__.py
 *  /usr/lib64/python3.3/site-packages/bcrypt/__init__.py
 * 
 * Package 'dev-python/bcrypt-1.0.2' NOT merged due to file collisions.
 * If necessary, refer to your elog messages for the whole content of the
 * above message.

These similarly named libs have gone and assigned identically named system 
installed files.  tilt.  This means that gentoo has to do some serious 
tampering so as to get the 2 separate libs installed into the one common gentoo 
system.

tilt

Without this tampering, gentoo is obliged to do some compromising.  We can have 
this or that installed under this or that condition or needed python 
implementation.

I think it's not the first time I have seen this, however the file collisions 
phenomenon raises the degree of difficulty or incompatibility to another level 
or 3.
The nice news is that the testsuite will pass all once setup to the individual 
req's stipulated by the source code.  At this point there ia a most minimal 
compromise option thought spells itself out for a final ebuild, however a 
compromise is a compromise is a compromise.

Original comment by del...@iinet.com.au on 29 Dec 2013 at 9:23

GoogleCodeExporter commented 9 years ago 
Oh yeah...  I forgot to mention in my bcrypt library description that the 
'bcrypt' package is meant to be a drop-in replacement for 'py-bcrypt', so they 
use a number of the exact same python files.  Which is why none of my test 
environments within tox.ini use both at the same time.

I'm not a gentoo user, so I'm not quite clear why you're trying to install both 
of them.  It shouldn't be needed for testing: if you're running tox, it should 
be handling its own installation of the libraries inside an isolated python 
virtualenv, without affecting what's installed on your system.  Whereas in an 
actual deployment, applications that use passlib are welcome to require either 
(or neither) of them... all the python bcrypt libraries are optional 
dependencies for passlib, it will use them if it finds them.

I hope the tox.ini suite isn't too much of a beast.  For my own release testing 
purposes, I tried to include a large variety of 
implementation-and-optional-dependency combinations... but that does make it 
require a bit of setup and cpu cycles for the full thing to run :)

- Eli

Original comment by elic@astllc.org on 29 Dec 2013 at 6:18

GoogleCodeExporter commented 9 years ago 
I'm not a gentoo user, so I'm not quite clear why you're trying to install both 
of them.

For what it's worth I can explain in a line or 2.  The explanation once given 
changes nothing though.  Here;

The use of tox, while not banned, is strongly discouraged and in practise 
rarely ever done in a python package ebuild.  Gentoo's portage has a rule that 
bans the importing of, or connecting to the network, in any way beyond the 
fetch phase of an emerge which is fundamentally a system install, == apt-get or 
rpm -i. 

"if you're running tox, it should be handling its own installation of the 
libraries inside an isolated python virtualenv, without affecting what's 
installed on your system"

This is what's called the perfect mis-match.  Basically gentoo's policies makes 
for "it's got to work by utilising and drawing from (importing and so forth) 
the system installed packages' files".  The use of tox 'conveniently' skirts 
this imposition.  As mis-matches go, that makes for perfection.

ok, how about a line or 20.

I hope the tox.ini suite isn't too much of a beast.

It is, without a doubt, a beast.  However all is not lost.  We @ gentoo have 
dealt before with such beasts, and are sure to have to do so again.  The thing 
about your testsuite, tox aside, is that it is very comprehensive. and that's a 
good thingy.  I've established in my own gentoo system that they do pass once 
given a lib combo to suite a given python 'impl'.  That it draws upon optional 
libraries for optional testing provides us with some room to move.  With this 
room, 'we' can make and invoke a compromise that will give a gentoo ebuild a 
worable and working test phase.

Thx for your solid explanations and support of pypy.

Original comment by del...@iinet.com.au on 29 Dec 2013 at 10:43

GoogleCodeExporter commented 9 years ago 
That does make sense, especially in hindsight. Thanks for explaining it out! 
Happy to help.

Original comment by elic@astllc.org on 30 Dec 2013 at 4:01