pip install stuck at version 0.8.17 in arch linux Python 3.9

[alexjarosch]

Hi Rolf, as pointed out earlier, when trying to install PSS with pip on manjaro (arch) linux with python 3.9 , pip only finds version 0.8.17, which is unexpected. To reproduce the result, one can set up a new virtual environment in a shell

python -m venv pss-dev
source pss-dev/bin/activate
pip install --upgrade pip
pip install planetary-system-stacker

it also does the same if I ignore the cached versions with

pip install --no-cache-dir planetary-system-stacker

I am on this as you can see, but I thought I start using our issue system

Alex

[alexjarosch]

I think it's caused by the setup.py script requiring python to be lower than 3.7:

https://github.com/Rolf-Hempel/PlanetarySystemStacker/blob/dfc066de6f14aeaf5422315b00d07e47c005fc88/setup.py#L39

and the PiPY build instructions

https://github.com/Rolf-Hempel/PlanetarySystemStacker/blob/dfc066de6f14aeaf5422315b00d07e47c005fc88/PyPI_Instructions.txt#L3-L7

I'll keep working on that when time allows.

Alex

[Rolf-Hempel]

Hi Alex,

I don't think that the requirements on the Python version can be the reason. When I build a new version, I don't use the file "setup.py", but only the ones for the different OS types. In your case (Linux), it's the "manylinux1..." file. There the only requirement on Pyhon is: python_requires='>=3.5', so Python 3.9 will do.

All the best, Rolf

[alexjarosch]

Hi Rolf,

true. If I build the PSS package locally and install it:

python3 setup_linux.py bdist_wheel --plat-name manylinux1_x86_64
pip install dist/planetary_system_stacker-0.8.25-py3-none-manylinux1_x86_64.whl

then it updates correctly. However if I remove PSS again and try to install the newest from the PyPI server

pip uninstall planetary-system-stacker
pip install planetary-system-stacker

it returns back to version 0.8.17

Could it be that something went wrong building the package for Linux? When I check the official PyPI server info for different versions

• v 0.8.17
• v 0.8.19
• and so on to
• v 0.8.29

then they all tell me that from version 0.8.19 onward the Python requirements are Python >=3.5, <3.9 Hence it does not update on a Python 3.9 machine.

Even though the linux setup file says https://github.com/Rolf-Hempel/PlanetarySystemStacker/blob/dfc066de6f14aeaf5422315b00d07e47c005fc88/setup_linux.py#L38 the MacOs setup file limits it down below 3.9 https://github.com/Rolf-Hempel/PlanetarySystemStacker/blob/dfc066de6f14aeaf5422315b00d07e47c005fc88/setup_macos.py#L38 Could it be that you (your scripts) build the linux version with the macos setup file?

All the best, Alex

[Rolf-Hempel]

Hi Alex,

The documentation file "PyPI_Instructions.txt" shows how I build the PyPI wheel files. You have access to the repository, so you can see what the scripts are doing. In the Linux script the condition on Python is "python_requires='>=3.5'".

Why don't you make experiments yourself on the PyPI test server (the instructions file above tells you how to do that). The question is if Python requirements on Windows carry over to the Python wheel. You could, for example, try to upload a version for Linux only, then the next version for Linux and Windows, and see if there is a difference.

I don't have Python 3.9 installed yet. On Ubuntu 20.04 LTS with Python 3.8.5 PSS installs and updates correctly.

All the best, Rolf

[alexjarosch]

Hi Rolf,

Sounds good to me. I'll report back.

All the best, Alex

[alexjarosch]

Hi Rolf,

did the tests on the PyPI test server. If I do

python3 setup_linux.py bdist_wheel --plat-name manylinux1_x86_64
python3 setup_windows.py bdist_wheel --plat-name win_amd64
python3 setup_macos.py bdist_wheel --plat-name macosx_10_6_intel
python3 -m twine upload --repository testpypi dist/

my test server project gets it right and does not mix up the versions. I can also install the package. So I can't reproduce the setup problem on the actual PyPI server. Hmm.

My guess is that since we use wheel and your linux system uses python 3.8.5, the PyPI server limits it down to python 3.8. Could that be.

Shall we consider building the linux wheel package on my machine for the next release?

All the best Alex

[Rolf-Hempel]

Hi Alex,

That is strange, indeed! I build the wheels (Windows, Linux, MacOS) on my Windows machine, where Python is 3.6.8. After that, I have no problems to install the Linux wheel on my Ubuntu 20.04 LTS with Python 3.8.5. If the Python version restriction came from the host Python version, that should not work any better than with Python 3.9.

The only difference I see is that I build the Windows wheel first, then Linux and MacOS, and you started with Linux. If PyPI took the Python restriction from the first uploaded wheel, and applied it to the other ones, this would explain the difference. But that would be rather stupid, wouldn't it?

I had a closer look at the files the setup procedure produces. In the pss.egg-info directory there is the file PKG-INFO. In this file the Python version restriction is set by the last whell which is produced. If one, therefore, creates the Linux wheel last, the condition is "Requires-Python: >=3.5" which would be fine with you. The condition "Requires-Python: >=3.5, <3.9" on the official PyPI server comes from the MacOS wheel which I produce last. It could well be that I inserted the <3.9 after PSS 0.8.17.

Well, this would explain everything if you had not produced the MacOS wheel last as well. So you should see the same restriction <3.9. What does your PKG-INFO file say?

Another point: I'm not sure any more that one can set different Python version restrictions for different wheels. So, if the "most liberal" setting (Linux) is produced last, I would guess that on Windows systems PSS would not refuse to install on a Python 3.8 (which would make trouble with Astropy). If this is really the case, I guess we can only set the "liberal" restrictions and tell users not to use Python 3.9 on MacOS and not Python 3.7 - 3.9 on Windows. :-(

All the best, Rolf

[alexjarosch]

Hi Rolf,

my PKG-INFO says Requires-Python: >=3.5, <3.9.

However, when I build and upload each version individually and remove the build, dist and egg dirs in between, the problem is gone. Like

python3 setup_linux.py bdist_wheel --plat-name manylinux1_x86_64
python3 -m twine upload --repository testpypi dist/planetary_system_stacker-0.8.25-py3-none-manylinux1_x86_64.whl
rm -r dist/ build/ planetary_system_stacker.egg-info/

python3 setup_windows.py bdist_wheel --plat-name win_amd64
python3 -m twine upload --repository testpypi dist/planetary_system_stacker-0.8.25-py3-none-win_amd64.whl
rm -r dist/ build/ planetary_system_stacker.egg-info/

python3 setup_macos.py bdist_wheel --plat-name macosx_10_6_intel
python3 -m twine upload --repository testpypi dist/planetary_system_stacker-0.8.25-py3-none-macosx_10_6_intel.whl
rm -r dist/ build/ planetary_system_stacker.egg-info/

this is rather unexpected. Maybe you can try to reproduce that from your side on the test server with an alpha pre release version. Then we could use that approach for the next release.

All the best Alex

[ghilios]

I thought we had issues with scikit being available for 3.9 in https://github.com/Rolf-Hempel/PlanetarySystemStacker/issues/34? Or is only Mac affected for that?

[Rolf-Hempel]

As far as I remember this was only on Mac. Otherwise Alex could not be able to run PSS on his Python 3.9 on Linux.

[alexjarosch]

Yes, no Problem here if I build it locally with

scikit-image 0.18.1
Python 3.9.1

installs fine and runs smoothly.

[Rolf-Hempel]

Hi Alex,

The solution you proposed does not work. I found out that the Linux requirements are set at the first upload of a given version. The requirements are taken from the last wheel uploaded at that time. Even if you delete the three directories locally, the next uploads to the server (for the same release tag) don't change that anymore.

In your case you uploaded the Linux version in first, so it is no surprise that it worked afterwards for your Linux computer. The problem is that on a Windows computer the same restrictions apply.

In my test I first uploaded the Windows version (only), followed by separate uploads for Linux and MacOS. On Linux it refused to install the wheel on Python 3.8.5 because of the Windows restriction.

The only solution in my view is to upload three different releases (0.8.30.1 for Windows, 0.8.30.2 for Linux, 0.8.30.3 for macOS).

All the best, Rolf

[Rolf-Hempel]

I now created and uploaded the three separate versions on PyPI.

[alexjarosch]

Good that this is fixed! Great work Rolf.