vphilippon
commented
7 years ago I looked a bit more into this, and I'm understanding a bit more of what's going on. I hope this will help find a good fix:
This call:
File "c:\python27\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2816, in _compute_dependencies
common = frozenset(reqs_for_extra(None))tries to gather all requirements for which there is either no marker, or for which the marker, when setting extra to None, evaluates as True.
As an example, given the following requirements:
Requires-Dist: requests (==2.13.0)
Requires-Dist: toto (==7.8.9); sys_platform == "win32"
Requires-Dist: bar (==4.2.1); extra == None
Requires-Dist: foo (==1.2.4); extra == '3.10'common would contain the 1st, 2nd and 3rd requirements.
I can see where the 1st one could come from: the install_requires list.
The second one, probably from extras_require: {':sys_platform == "win32"': ['toto==7.8.9']}
The 3rd one would be from something like extras_require: {None: ['bar==4.2.1']}, which I'm not sure should be part of common, or is even possible (I claim here my right to be wrong and to not have a clue).
Now, as I pointed out in my initial comment, the issue happens with the 4th requirements.
When evaluating the marker of that requirements, we call _eval_op.
And on this line:
spec = Specifier("".join([op.serialize(), rhs]))
We succed, because: op.serialize() -> "==", rhs -> "3.10"
So we call this line of code:
return spec.contains(lhs)
And we get an exception, because: lhs -> None,
and trying to parse None with a regex to create a Version object fails.
So, it looks like the issue is from either:
pkg_resources, which shouldn't be calling an equivalent ofreq.marker.evaluate({'extra': None}), although it does not seem wrong to me (considering, once again, my little knowledge).packaging, which should either handle theNonein_eval_op, or inSpecifier.contains, or use some additionnal logic before usingSpecifier.contains, such as making sure thatlhscan be parsed as aVersion, such that evaluating this with theVersionlogic is what is really intended.
While this seems outside of the hand of pip itself, could someone knowledgeable (@dstufft ?) give me some pointers on which one I should go to and open an issue? I have a feeling that Specifier.contains in packaging should be handling None nicely, but I'd really like a second opinion.
Thanks to whoever will take the time to read all this and reply :) .
dstufft
pfmoore
pradyunsg
q0w
webknjaz
Description:
I have a package which provides extras. The extras keys happen to have the format of PEP440 versions, such as
3.2.1or3.0. When installing the package usingpip install, I end up with an exception (see in the "What I've run" section).Note: While this occured using
pip, I'm not exactly sure who is beign faulty (pip, setuptools, pkg_resources, packaging?)What I've run:
Here's the setup.py to make a package that reproduces the issue:
Then, make a wheel:
Then, install the package (not even using the extra):
And the result:
What I figured out
I went around the code in the stack trace and managed to figure what's happening:
The
TypeErrorcomes from trying to parseNoneas apackaging.Versionobject. TheNonecomes fromDistInfoDistribution._compute_dependencies, with this call:common = frozenset(reqs_for_extra(None)).Later, during the markers evaluation in
packaging.markers, at line 185, we have this function:As
rhsis the extra key, a PEP440 version (3.2.1), theSpecifiercreation succeeds, and we callreturn spec.contains(lhs). In this case,lhsis theNoneI pointed out earlier, which cause theTypeErrorwhen trying to parse it as a version. Note that ifrhswas not a version-like string (such asfoobarorsecurity), theSpecifiercreation would fail, (I think) and we would executeoper = _operators.get(op.serialize())andreturn oper(lhs, rhs)instead.How to fix
I'm not sure. I feel like the logic in
_eval_opis not precise enough, or maybe the marker evaluation is not meant to be used this way. I'm not familiar enough with pip/pkg_ressources/packaging structure and interaction to tell what's going to happen if I change any of this.I'd need the input of someone with a lot more knowledge about the project to help sort this out.
I tried making this as clear as possible, but don't hesitate to ask for details.