search

mk-fg / pretty-yaml

PyYAML-based module to produce a bit more pretty and readable YAML-serialized data
Do What The F*ck You Want To Public License
135 stars 29 forks source link

23.5.9 from pypi installs empty directory #43

Closed nieder closed 1 year ago

nieder commented 1 year ago

When trying to build 23.5.9, the build installs an empty directory with just an UNKNOWN-0.0.0-py3.10.egg-info subdir.

On macOS (though this shouldn't matter):

$ /sw/bin/python3.10 setup.py build
running build
$ PYTHONPATH=/sw/build.build/pyaml-py310-23.5.9-1/pyaml-23.5.9/build/lib /sw/bin/python3.10 -m unittest || exit 2
......................
----------------------------------------------------------------------
Ran 22 tests in 0.125s

OK
$ /sw/bin/python3.10 setup.py install --root=/sw/build.build/root-pyaml-py310-23.5.9-1
running install
/sw/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running build
running install_egg_info
running egg_info
creating UNKNOWN.egg-info
writing UNKNOWN.egg-info/PKG-INFO
writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
writing top-level names to UNKNOWN.egg-info/top_level.txt
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'COPYING'
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
Copying UNKNOWN.egg-info to /sw/build.build/root-pyaml-py310-23.5.9-1/sw/lib/python3.10/site-packages/UNKNOWN-0.0.0-py3.10.egg-info
running install_scripts

The commands finish cleanly, but the installation directory is empty except for those files inside the UNKNOWN.egg-info directory. I've also tried with a build based process, but the results are the same. pyaml-21.10.1 built and installed fine w/ a full tree.

mk-fg commented 1 year ago

I think issue is likely with older setuptools that you might be using there - it seem to work here with setuptools-68.0.0.

And given giant banner warning that it produces these days (see below, though you have similar one in your output), I'd suggest to avoid using setuptools and "python setup.py" command entirely, if that is an option. (e.g. maybe use "python -m venv" if you can, pretty sure that should always pull latest python packaging tools, avoiding the issue)

If it's for old project, with old build process and dependencies, maybe just stick with older pyaml version as well, as it'd be more likely to work with any other stuff installed there (e.g. python 3.10 and other modules that might be using it) - 23.x.x has couple potentially-breaking changes that you might avoid that way.

Proper fix might be restoring setup.py file from earlier 21.x version, which was nuked in 6b9113e4616316c50c6b7a98c256eb46b2ffbd83 , but I don't think it's common enough to use older setuptools on a system without venv or pip and such, to maintain setup.py for.

Though also, gotta admit that python packaging tools always look confusing for me, with 10 ways of doing same thing in there, so maybe I don't understand the issue and what should be used these days either.

% curl -OL https://files.pythonhosted.org/packages/e5/c1/ad25b528ba5e7039ec0939330d6ddb05212033f5fe0041e90daf5bb3b0da/pyaml-23.5.9.tar.gz
% tar -xf pyaml-23.5.9.tar.gz
% cd pyaml-23.5.9
% python setup.py install --root=/tmp/test

/usr/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:72: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
  config = read_configuration(filepath, True, ignore_option_errors, dist)
validate_pyproject.api.load_builtin_plugin defines `tool.distutils` schema
validate_pyproject.api.load_builtin_plugin defines `tool.setuptools` schema
running install
/usr/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
running build
running build_py
creating build
creating build/lib
creating build/lib/pyaml
copying pyaml/__init__.py -> build/lib/pyaml
copying pyaml/__main__.py -> build/lib/pyaml
creating build/lib/pyaml/tests
copying pyaml/tests/__init__.py -> build/lib/pyaml/tests
copying pyaml/tests/test_dump.py -> build/lib/pyaml/tests
running egg_info
writing pyaml.egg-info/PKG-INFO
writing dependency_links to pyaml.egg-info/dependency_links.txt
writing requirements to pyaml.egg-info/requires.txt
writing top-level names to pyaml.egg-info/top_level.txt
reading manifest file 'pyaml.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'COPYING'
writing manifest file 'pyaml.egg-info/SOURCES.txt'
warning: build_py: byte-compiling is disabled, skipping.

running install_lib
creating /tmp/test
creating /tmp/test/usr
creating /tmp/test/usr/lib
creating /tmp/test/usr/lib/python3.11
creating /tmp/test/usr/lib/python3.11/site-packages
creating /tmp/test/usr/lib/python3.11/site-packages/pyaml
copying build/lib/pyaml/__init__.py -> /tmp/test/usr/lib/python3.11/site-packages/pyaml
copying build/lib/pyaml/__main__.py -> /tmp/test/usr/lib/python3.11/site-packages/pyaml
creating /tmp/test/usr/lib/python3.11/site-packages/pyaml/tests
copying build/lib/pyaml/tests/__init__.py -> /tmp/test/usr/lib/python3.11/site-packages/pyaml/tests
copying build/lib/pyaml/tests/test_dump.py -> /tmp/test/usr/lib/python3.11/site-packages/pyaml/tests
warning: install_lib: byte-compiling is disabled, skipping.

running install_egg_info
Copying pyaml.egg-info to /tmp/test/usr/lib/python3.11/site-packages/pyaml-23.5.9-py3.11.egg-info
running install_scripts

% find /tmp/test

/tmp/test
/tmp/test/usr
/tmp/test/usr/lib
/tmp/test/usr/lib/python3.11
/tmp/test/usr/lib/python3.11/site-packages
/tmp/test/usr/lib/python3.11/site-packages/pyaml-23.5.9-py3.11.egg-info
/tmp/test/usr/lib/python3.11/site-packages/pyaml-23.5.9-py3.11.egg-info/top_level.txt
/tmp/test/usr/lib/python3.11/site-packages/pyaml-23.5.9-py3.11.egg-info/requires.txt
/tmp/test/usr/lib/python3.11/site-packages/pyaml-23.5.9-py3.11.egg-info/dependency_links.txt
/tmp/test/usr/lib/python3.11/site-packages/pyaml-23.5.9-py3.11.egg-info/SOURCES.txt
/tmp/test/usr/lib/python3.11/site-packages/pyaml-23.5.9-py3.11.egg-info/PKG-INFO
/tmp/test/usr/lib/python3.11/site-packages/pyaml
/tmp/test/usr/lib/python3.11/site-packages/pyaml/tests
/tmp/test/usr/lib/python3.11/site-packages/pyaml/tests/test_dump.py
/tmp/test/usr/lib/python3.11/site-packages/pyaml/tests/__init__.py
/tmp/test/usr/lib/python3.11/site-packages/pyaml/__main__.py
/tmp/test/usr/lib/python3.11/site-packages/pyaml/__init__.py
nieder commented 1 year ago

I get the same failure type when using build/wheel to install:

$ /sw/bin/python3.10 -m build --wheel --no-isolation
* Building wheel...
running bdist_wheel
running build
installing to build/bdist.macosx-10.14-x86_64/wheel
running install
running install_egg_info
running egg_info
creating UNKNOWN.egg-info
writing UNKNOWN.egg-info/PKG-INFO
writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
writing top-level names to UNKNOWN.egg-info/top_level.txt
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
listing git files failed - pretending there aren't any
listing git files failed - pretending there aren't any
reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'COPYING'
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
Copying UNKNOWN.egg-info to build/bdist.macosx-10.14-x86_64/wheel/UNKNOWN-0.0.0-py3.10.egg-info
running install_scripts
adding license file "COPYING" (matched pattern "COPYING*")
creating build/bdist.macosx-10.14-x86_64/wheel/UNKNOWN-0.0.0.dist-info/WHEEL
creating '/sw/build.build/pyaml-py310-23.5.9-1/pyaml-23.5.9/dist/tmpxo9_yx4p/UNKNOWN-0.0.0-py3-none-any.whl' and adding 'build/bdist.macosx-10.14-x86_64/wheel' to it
adding 'UNKNOWN-0.0.0.dist-info/COPYING'
adding 'UNKNOWN-0.0.0.dist-info/METADATA'
adding 'UNKNOWN-0.0.0.dist-info/WHEEL'
adding 'UNKNOWN-0.0.0.dist-info/top_level.txt'
adding 'UNKNOWN-0.0.0.dist-info/RECORD'
removing build/bdist.macosx-10.14-x86_64/wheel
Successfully built UNKNOWN-0.0.0-py3-none-any.whl

$ /sw/bin/python3.10 -m unittest -vv
< ALL TESTS PASS >

$ find /tmp/build-root
/tmp/build-root/
/tmp/build-root/sw/
/tmp/build-root/sw/lib/
/tmp/build-root/sw/lib/python3.10/
/tmp/build-root/sw/lib/python3.10/site-packages/
/tmp/build-root/sw/lib/python3.10/site-packages/UNKNOWN-0.0.0.dist-info/
/tmp/build-root/sw/lib/python3.10/site-packages/UNKNOWN-0.0.0.dist-info/RECORD
/tmp/build-root/sw/lib/python3.10/site-packages/UNKNOWN-0.0.0.dist-info/WHEEL
/tmp/build-root/sw/lib/python3.10/site-packages/UNKNOWN-0.0.0.dist-info/COPYING
/tmp/build-root/sw/lib/python3.10/site-packages/UNKNOWN-0.0.0.dist-info/top_level.txt
/tmp/build-root/sw/lib/python3.10/site-packages/UNKNOWN-0.0.0.dist-info/METADATA

Is pyaml using setuptools under the hood even when using the more modern build/wheel frontend? Most other pymods I've set up (including older pyaml-21.10.1) work with either setup.py or build/wheel, which is why I thought the problem/incompatibility might be somewhere in what the new pyaml is expecting. I've also never seen the UNKNOWN-0.0.0 tags, so something clearly is missing. But you're right, it near impossible these days to know what python packaging tools to use and how they're supposed to behave. And my setuptools is older because newer versions break the numpy we have, and newer numpy is failing various tests :(

In that install you sent to /tmp/test, can you post the contents of requires.txt and dependency_links.txt ? Maybe there's some dep there I'm missing that's not obvious. Thanks.

mk-fg commented 1 year ago

I've also never seen the UNKNOWN-0.0.0 tags

I think it might be missing version and package name in old setup() call of setuptools, which is how pypa documents making fallback, which clearly doesn't work for older versions of something that does not yet parse those in pyproject.toml

Guess restoring it would still be simplier than having to know about what -m build is, how that differs between multiple python releases, and other minor incompatibilities :)

mk-fg commented 1 year ago

In that install you sent to /tmp/test, can you post the contents of requires.txt and dependency_links.txt ? Maybe there's some dep there I'm missing that's not obvious. Thanks.

requires.txt:

PyYAML

[anchors]
unidecode

dependency_links.txt seem to only have two empty lines in it.

Don't think these should be important for how setuptools and such work, as presumably these are package dependencies, which tools that build/install package generate by parsing its metadata, and shouldn't have/need in advance.

mk-fg commented 1 year ago

Restored all setup.py keys by parsing those from pyproject.toml in 1efa7da / 23.7.0, so hopefully it should work same as before on its own, even if setuptools knows nothing about pyproject.toml yet.

Let me know if it still does not, as I haven't really reproduced your setup and issue, and maybe it's about something other than keywords in setup.py entirely.

nieder commented 1 year ago

That worked. The 23.7.0 build process now uses the name/version keys and the final build has all the expected files. Thanks for taking time for this!