Missing python_requires on sdist distributions

[Thrameos]

@marscher It appears that we are missing a specification in our current releases.

https://github.com/pantsbuild/pex/issues/1202

This is a problem as py2 packages are pulling the latest JPype and then attempting to build. Not sure what is a good idea to rerelease the old releases to remove the sdist to fix this. Though this seems like a big pain to clean up.

[marscher]

Honestly I've never seen any usage of this specifier before.

I guess we should use pep345 for this purpose, hoping build tools will respect it. https://www.python.org/dev/peps/pep-0345/#requires-python

This would involve adding the metadata file to every source archive prior 0.7.5

Since Python2 is dead since a year now (and has been deprecated four years ago) I do not see why we should spent (lots of) time on this one. People downstream can pin JPype1<0.7.5, if they still want to ride on dead snakes.

Another idea would be to redirect people to the recent version by yanking the older ones (https://www.python.org/dev/peps/pep-0592/). Pinning old version would still work, but dependency solvers would prefer the un-yanked versions.

[marscher]

yanking wouldn't solve this, because the newer versions are not compatible with Python2 -- so it makes no sense to redirect Python2 users there. I'd suggest projects to pin the version. Another option would be to built wheels for Python2 (for <0.7.5) for the sake of the referenced project, but that seems like lots of hassle as well, for supporting outdated and buggy code.

[marscher]

Marking all releases onward 0.7.2 as Python3 only will make re-upload 10 release archives (in a different format, so that PyPI will accept them) containing the new flag in setup.py or metadata. Seems doable, but should be tested.

[Thrameos]

I think we only need to reupload the last patch of each series. You could then yank the earlier patch releases. So we only have 1.0, 1.1, and 1.2 to fix.

[marscher]

Ah this makes sense :) Thanks for pointing this out.

[marscher]

Do you agree with this Python3 version pinning in PKGINFO?

Requires-Python: ">=3.3, <4"

[marscher]

Or should it be 3.5, since we have not tested for 3.3 and 3.4, if I recall correctly?

[Thrameos]

I think 3.5 is correct given that is all we test.

[marscher]

I just tried the suggest approach from https://packaging.python.org/guides/dropping-older-python-versions/ but it seems pip is just ignoring it, when installing the source archive directly. So we would have to rely upon PyPI (re-)parsing this (updated) metadata from the modified source archive and do not offer it again for py2 users (installing with pip). I will quickly check on the testing instance of pypi if that is the case. If not, I do not see any chance to fix the upstream issue.

[Thrameos]

Sounds like the best we can attempt. If that doesnt work then it will have to be a do not fix.

[marscher]

Editing the PKG-INFO works, but yanking older patches (or dev versions) does not hinder pip to go for the yanked file (at least if it is the only remaining version). As test-pypi and pypi are the same software on separate servers, I conclude that this approach is only working, when I edit all python3 releases...

[marscher]

https://github.com/pypa/warehouse/issues/2700#issuecomment-769113148

[Thrameos]

@marscher I am a bit confused about the request for adding Python-Requires: >=3.5, <4 to version prior to 0.7.1

If you do not intend to integrate such a feature (which I can totally understand), I'd be delighted, if somebody with super-sql-rights could update the python-require string for jpype1 (https://github.com/jpype-project/jpype, https://pypi.org/manage/project/jpype1/releases/ ) to ">=3.5, <4" for all releases <0.7.1

Versions prior to 0.7.0 work fine for Python 2,7 (some even Python 2.6) so should they have Python-Requires: >=2.6, <4. In which case we shouldn't need to modify them.

It would be great if Python pypa checked to see if a tag like this was missing and gave a warning. Surely we are not the only ones that have ancient versions that are missing specifications.

One other trivia question..... Why is this project named JPype1 on pypi?

[marscher]

I meant for all releases >0.7.1, since these are not compatible with Python2 anymore. There was this one recent version, which caused a bug on a certain Python version, do you recall which one that was, so I can directly restrict installing this one on this version by specifying it in python_requires?

I think it was named that way by @originell, because jpype was already on PyPI for the outdated original software.

[Thrameos]

Okay I think the change over point for Py3 was 0.7.2. 0.7.0 and 0.7.1 should be safe for Py2. As far as specific version bugs there was a global variable created outside of the GIL. The line is

native/python/pyjp_class.cpp:

static PyObject* classMagic = PyDict_New();

This line breaks Python 3.9 on because they updated the load procedure which now releases the GIL during the library initialization phase. The old behavior was buggy. The bug only affects certain architectures. For linux it is a transient memory error because it borrows a dict from outside the GIL which will cause an issue later during shutdown, but I having been able to replicate that particular failure mode. Windows is a hard crash depending on the state of the available Python objects as if there is no free dicts available it will allocate a new memory block outside of the GIL.

Checking version history.
0.7.5 not present 1.0.0 present 1.1.0 present 1.2.0 present 1.2.1 fixed

So version 1.0.x and 1.1.x should have the bug fix backported if we want to be able to run on Python 3.9. It is only a 3 line patch (remove the initialization and then add it when the module is created).

[Thrameos]

Seems like the conflict with "jpype1" and "jpype" has gone away. At least I don't see anything at that address from a simple search

https://pypi.org/project/JPype/

Though there are still a good number dead packages in the search list from earlier forks.

https://pypi.org/project/JPype1-py3/ https://pypi.org/project/jpypex/

Don't suppose there is some way that we can rename to jpype and leave jpype1 as an alias? I just don't want someone to name squat us by registering jpype like potentially some random fork or the guy that is trying to unify all bridge codes from some very early version

https://pypi.org/project/ha_JPype1/ https://pypi.org/project/jtypes.jpype/

If pip installing jpype installed some other software is would certainly add to a world of problems for us.

[marscher]

It might have gone in the meantime, which doesn't mean that anyone would be able to re-register that name again for the reasons you listed. Unfortunately these kind of naming attacks happen all the time on PyPI and if users don't research prior installing random software on their computers, nobody can help them.

[Thrameos]

I tried to run a few of the scripts that proport to check to see if a name is available and was unable to get any of them to function enough to test if "jpype" is currently in taken. It is reserved and can't be used then all is fine. Currently pip install jpype gives a 404.

I looked over the relevant pep and there really isn't anything about what happens if the main line of a project is abandoned and was removed and the fork is now the main which is sort of where we are today. It would seem to imply that we could claim the jpype name if it were abandoned but it isn't clear we could put anything there but an alias package which requires jpype1 which may be afoul of the name squatting probation. Maybe I am the only one that finds the name jpype1 to be silly when at some point it will be 2.0 and 3.0 and it was jpype1 when it was 0.5. Of course maybe the ship has sailed as it would have been best to take the jpype name when we finally got to 1.0 or when we had abandoned Py2.

https://www.python.org/dev/peps/pep-0541/

If is is available we are sort of open for getting name squat.

[pelson]

Perhaps register jpype ASAP, and make it depend on jpype1 (or the inverse)? Or even just raise an exception in the setup.py of the name you don't want. There is no legitimate reason for having some other package called jpype in the community which isn't this jpype.

[pelson]

Indeed, I took some unilateral action and registered the name (adding you both as owners) with the following setup.py:

import sys

# XXX This if statement lets you actually build the package
if not 'sdist' in sys.argv:
    sys.exit("""
=========================================

Please install the `JPype` with ``pip install jpype1``

=========================================

""")

from distutils.core import setup

setup(
    name="jpype",
    version="0.0",
    description="Unsupported alias for JPype1 (https://pypi.org/project/JPype1/)"
)

When you pip install this:

$ pip install jpype
Collecting jpype
    ERROR: Command errored out with exit status 1:
     command: /home/pelson/conda2/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3qizookx/jpype/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3qizookx/jpype/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-9zs61xw2
         cwd: /tmp/pip-install-3qizookx/jpype/
    Complete output (8 lines):

    =========================================

    Please install the `JPype` with ``pip install jpype1``

    =========================================

    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

If this is a problem, I have no issue removing the project from pypi - just let me know.

It would be easy to transition this such that jpype depends on jpype1 or visa-versa.

[Thrameos]

I think @marscher should have the decision on this. It sort of looks like @originell was leaving the base name open in case non-forked version was going forward, but at this point that seems very unlikely. So long as install jpype points to the current active fork (this one or perhaps in the future some other if this gets abandoned) it seems acceptable but I am not sure about the name squatting rules.

[marscher]

I'am fine with transitioning to the "jpype"naming. But I guess this will cause a lot of trouble for dependent projects, as there is no such such thing as aliasing on PyPI (as far as I know) and automatic dependency pulling would require micro patch releases for all versions we have published so far.

[ShaheedHaque]

automatic dependency pulling would require micro patch releases for all versions we have published so far.

Well, since jpype was dead anyway, couldn't you assert support only for versions going forward? You could even be more aggressive and assert support only for "current"...