Closed LukasVik closed 3 years ago
So, the issue seems to be the following warning:
Using legacy 'setup.py install' for vunit-hdl, since package 'wheel' is not installed.
I now added wheel
to the VUnit images (https://github.com/ghdl/docker/commit/afb90b627e0007789d2aa4174679ceed8c2de474). It seems that the problem was solved for "stable" images, but not for *-master
:
# docker run --rm -it ghdl/vunit:mcode pip3 list
Package Version
---------- -------
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:mcode-master pip3 list
Package Version
---------- -------
pip 20.2.2
setuptools 50.0.0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:llvm pip3 list
Package Version
---------- -------
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:llvm-master pip3 list
Package Version
---------- -------
pip 20.2.2
setuptools 50.0.0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:gcc pip3 list
Package Version
---------- -------
colorama 0.4.3
gcovr 4.2
Jinja2 2.11.2
lxml 4.5.2
MarkupSafe 1.1.1
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:gcc-master pip3 list
Package Version
---------- -------
gcovr 4.2
Jinja2 2.11.2
lxml 4.5.2
MarkupSafe 1.1.1
pip 20.2.2
setuptools 50.0.0
wheel 0.35.1
That is, pip3 install vunit_hdl
works now, but python3 setup.py install
is still failing.
Unfortunately, I really don't understand the details of pip packages and the several mechanisms to install Python packages. Any guess?
I tried the same thing in the
python:3-slim-buster
(upon whichghdl/vunit:gcc-master
is based?
No, ghdl/vunit:*
images are based on ghdl/ghdl:buster-*
images, which are based on debian:buster-slim
.
OTOH, image python:3-slim-buster
is used as a base for ghdl/ext:ls-debian
. So, ghdl/ext:ls-vunit
is based indirectly on python:3-slim-buster
. It seems that wheel
is/was installed in python:3-slim-buster
, by default. Surprisingly, although python3 setup.py install
is used (https://github.com/ghdl/docker/blob/master/gui.dockerfile#L25), it works:
# docker run --rm -it ghdl/ext:ls-vunit pip3 list
Package Version
---------- --------
attrs 20.1.0
colorama 0.4.3
pip 20.2.2
pyghdl 1.0.dev0
setuptools 49.3.1
vunit-hdl 4.4.1rc0
wheel 0.34.2
Some more info: when installed through python3 setup.py install
, VUnit and colorama are indeed installed in /usr/lib/python3.7/site-packages/
:
# ls /usr/lib/python3.7/site-packages/
colorama-0.4.3-py3.7.egg/ easy-install.pth vunit_hdl-4.4.1rc0-py3.7.egg/
# cat /usr/lib/python3.7/site-packages/easy-install.pth
./vunit_hdl-4.4.1rc0-py3.7.egg
./colorama-0.4.3-py3.7.egg
But pip seems not to find those. Moreover, /usr/lib/python3/dist-packages/pip
exists.
Conversely, in ghdl/ext:ls-vunit
:
/# ls /usr/local/lib/python3.8/site-packages/
README.txt colorama-0.4.3-py3.8.egg pip setuptools-49.3.1.dist-info
__pycache__ distutils-precedence.pth pip-20.2.2.dist-info vhdl_langserver
_distutils_hack easy-install.pth pkg_resources vunit_hdl-4.4.1rc0-py3.8.egg
attr easy_install.py pyghdl-1.0.dev0.dist-info wheel
attrs-20.1.0.dist-info libghdl setuptools wheel-0.34.2.dist-info
Now, pip3 install .
is used instead of python3 setup.py install
. That should fix all the images:
# docker run --rm -it ghdl/vunit:mcode pip3 list
Package Version
---------- -------
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:mcode-master pip3 list
Package Version
---------- --------
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.1rc0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:llvm pip3 list
Package Version
---------- -------
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:llvm-master pip3 list
Package Version
---------- --------
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.1rc0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:gcc pip3 list
Package Version
---------- -------
colorama 0.4.3
gcovr 4.2
Jinja2 2.11.2
lxml 4.5.2
MarkupSafe 1.1.1
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
# docker run --rm -it ghdl/vunit:gcc-master pip3 list
Package Version
---------- --------
colorama 0.4.3
gcovr 4.2
Jinja2 2.11.2
lxml 4.5.2
MarkupSafe 1.1.1
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.1rc0
wheel 0.35.1
@LukasVik, please, let me know whether it works for you. If so, I think we can close this issue 😃
Okay great, that looks promising. I will try it out tomorrow at work (I don’t have a proper computer at home at the moment unfortunately). Thank you for the effort!
mån 31 aug. 2020 kl. 17:26 skrev eine notifications@github.com:
Now, pip3 install . is used instead of python3 setup.py install. That should fix all the images:
docker run --rm -it ghdl/vunit:mcode pip3 list
Package Version
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
docker run --rm -it ghdl/vunit:mcode-master pip3 list
Package Version
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.1rc0
wheel 0.35.1
docker run --rm -it ghdl/vunit:llvm pip3 list
Package Version
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
docker run --rm -it ghdl/vunit:llvm-master pip3 list
Package Version
colorama 0.4.3
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.1rc0
wheel 0.35.1
docker run --rm -it ghdl/vunit:gcc pip3 list
Package Version
colorama 0.4.3
gcovr 4.2
Jinja2 2.11.2
lxml 4.5.2
MarkupSafe 1.1.1
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.0
wheel 0.35.1
docker run --rm -it ghdl/vunit:gcc-master pip3 list
Package Version
colorama 0.4.3
gcovr 4.2
Jinja2 2.11.2
lxml 4.5.2
MarkupSafe 1.1.1
pip 20.2.2
setuptools 50.0.0
vunit-hdl 4.4.1rc0
wheel 0.35.1
@LukasVik https://github.com/LukasVik, please, let me know whether it works for you. If so, I think we can close this issue 😃
— You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/ghdl/docker/issues/31#issuecomment-683849541, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOEO64AX4ZJDUGIETLTGQDSDO6I5ANCNFSM4QQQMA5Q .
Good morning,
Indeed it does work well now. Thank you so much @eine!
Looking at the issue I still find it a little funny. I'm not an expert at pip, but there must be some mechanism that is broken in there. Where it supposedly installs a package, the files are there in the file tree, but it has not "registered" that it is installed. If pip truly needs wheel
in order to install packages, it should signal an error instead of the warning we got.
I googled the issue yesterday and I could not find anyone else with the same issue. And it also does not appear to happen in all python environments. On my ubuntu box I could install vunit_hdl
just fine without wheel, and also in the python:3-slim-buster
image after doing pip uninstall wheel
.
Just to note, I don't think it has anything to do with the vunit_hdl
package specifically. I observed the same issue with a python package named wrapt
and indeed the tsfpga
package.
Anyway, the fix/workaround works and that is all that matters I guess. I feel no need to dig deeper into the issue. Something seems a little sketchy but it works now! Closing this issue. Thanks again @eine!
@LukasVik, just for the sake of knowledge, let me drop my thoughts too:
I think we can blame setuptools
. Until two days ago, v49.6.0 was used (https://github.com/ghdl/docker/runs/1048161781?check_suite_focus=true#step:3:337). Yesterday, v50.0.0 was published (https://github.com/ghdl/docker/runs/1049962487?check_suite_focus=true#step:3:337). I don't see an explicit connection with the changes (https://github.com/pypa/setuptools/blob/master/CHANGES.rst), but that seems to be the case: pypa/setuptools#2350, pypa/setuptools#2232.
My poor understanding is that some commands create packages, others upload them, and others install them.
pyproject.toml
, instead of tox.ini
. However, that was two weeks ago.sdist
package (which I assume that is NOT installed), and it is pushed with twine: https://github.com/VUnit/vunit/blob/master/.github/workflows/push.yml#L164-L176.However, I don't understand what's the role of pip there. Also, I am not familiar with easy_install
. Using python3 setup.py install
seems to be "the easy_install/Egg way of doing things", while pip uses wheel. See https://packaging.python.org/discussions/pip-vs-easy-install/ and https://packaging.python.org/discussions/wheel-vs-egg/. Still, as you say, it is funny that this was working for months/years, and it broke yesterday without an explicit warning. I mean, the warning was shown by tox long before, not by setuptools.
On my ubuntu box I could install
vunit_hdl
just fine without wheel, and also in thepython:3-slim-buster
image after doingpip uninstall wheel
.
Is it possible that both your Ubuntu box and python:3-slim-buster
have Python >=3.8
? I'm thinking that this might be an issue with 3.7, 3.6... But I did not carefully read pypa/setuptools issues yet.
There seems to be an issue with python packages in
ghdl/vunit:gcc-master
, and possibly/probably others. The vunit package does not appear to be installed in the python environment:This issue appeared only recently. CI pipelines started failing in the tsfpga project after about 06:00 GMT+2 this morning.
To me it looks like some fatal python/pip problem is present:
So the vunit_hdl package does not appear to be installed according to pip. And even after a manual install it is not listed, and can not be imported.
I tried the same thing in the
python:3-slim-buster
(upon whichghdl/vunit:gcc-master
is based? I had a hard time following the scripts) and the problem was not present: