0.3.3: pep517 based build fails

[kloczek]

Looks like it something wrong when pep517 build is used.

+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting dependencies for wheel...
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
    main()
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 345, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
    return hook(config_settings)
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
    return self._get_build_requires(
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
    self.run_setup()
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 267, in run_setup
    super(_BuildMetaLegacyBackend,
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 158, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 2, in <module>
    from sphinxcontrib import asciinema as pkg
ImportError: cannot import name 'asciinema' from 'sphinxcontrib' (/usr/lib/python3.8/site-packages/sphinxcontrib/__init__.py)

ERROR Backend subproccess exited when trying to invoke get_requires_for_build_wheel

[divi255]

I can't repeat the above with pip3 install --use-pep517 .

Please specify what Python version you are using and what tool/mod for building/installing

[kloczek]

Please specify what Python version you are using and what tool/mod for building/installing

Python 3.8.12. Exact tool/mod is in firs line of quoted log (build module). Additionally if you are using just pip3 install --use-pep517 you are using doing build in isolated venv in which are downloaded from public network necessary .whl archives. Here is as well exact list of modules with versions installed in build env:

Package            Version
------------------ -------------------
appdirs            1.4.4
attrs              21.4.0
build              0.7.0
chardet            4.0.0
cycler             0.11.0
dbus-python        1.2.18
distro             1.6.0
extras             1.0.0
fixtures           3.0.0
fonttools          4.28.5
gpg                1.16.0-unknown
importlib-metadata 4.10.0
iniconfig          1.1.1
kiwisolver         1.3.2
libcomps           0.1.18
lit                13.0.0
lxml               4.7.1
Mako               1.1.6.dev0
Markdown           3.3.6
MarkupSafe         2.0.1
matplotlib         3.5.1
meson              0.60.3
numpy              1.22.0
olefile            0.46
packaging          21.3
pbr                5.8.0
pep517             0.12.0
Pillow             9.0.0
pip                21.3.1
pluggy             1.0.0
py                 1.11.0
Pygments           2.11.2
PyGObject          3.42.0
pyparsing          3.0.6
pytest             6.2.5
python-dateutil    2.8.2
rpm                4.17.0
scour              0.38.2
setuptools         60.1.0.post20211225
six                1.16.0
smartypants        2.0.1
testtools          2.5.0
toml               0.10.2
tomli              2.0.0
typogrify          2.0.7
wheel              0.37.1
zipp               3.7.0

[divi255]

# python3 --version
Python 3.8.10
#/usr/bin/python3 -sBm build -w --no-isolation
* Getting dependencies for wheel...
running egg_info
[skipped]
adding 'sphinxcontrib.asciinema-0.3.3.dist-info/METADATA'
adding 'sphinxcontrib.asciinema-0.3.3.dist-info/WHEEL'
adding 'sphinxcontrib.asciinema-0.3.3.dist-info/namespace_packages.txt'
adding 'sphinxcontrib.asciinema-0.3.3.dist-info/top_level.txt'
adding 'sphinxcontrib.asciinema-0.3.3.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built sphinxcontrib.asciinema-0.3.3-py3-none-any.whl

still can't repeat it. any ideas how to reproduce? (a clean system was used)

[kloczek]

still can't repeat it. any ideas how to reproduce? (a clean system was used)

Please try with setuptools > 60.

[divi255]

builds fine with 60.5.0 (latest) and 60.1.0 (probably yours). I can't find the build 60.1.0.post20211225 to check it

[kloczek]

By default setuptools adds current date to version so this is why you see version with post20211225. I've added to my setuptoos build below patch

python-setuptools-remove_use_tag_build_and_tag_date_from_setup.cfg.patch                                                                          331/331               100%
--- a/setup.cfg~        2022-01-17 00:40:51.000000000 +0000
+++ b/setup.cfg 2022-01-21 09:45:23.629543603 +0000
@@ -151,9 +151,5 @@
        depends.txt = setuptools.command.egg_info:warn_depends_obsolete
        dependency_links.txt = setuptools.command.egg_info:overwrite_arg

-[egg_info]
-tag_build = .post
-tag_date = 1
-
 [sdist]

to remove that. Neveretheless it is no relevant to this case. Just retested with setuptools 62.0.0 and still build ends like I've reported.

[ziotom78]

I can reproduce this bug using poetry:

$ mkdir test-asciinema
$ cd test-asciinema
$ poetry init --name "test-asciinema" --description="test" --author="Myself" --dependency=sphinx --dependency=sphinxcontrib.asciinema --license=GPL2 --no-interaction
Using version ^5.1.1 for Sphinx
Using version ^0.3.5 for sphinxcontrib.asciinema
$ poetry update                                                                                                                                        (base) ─╯
# Bang!

The error message produced by poetry is quite long, but it points to the same error message (cannot import name 'asciinema' from 'sphinxcontrib'):

Creating virtualenv test-asciinema-GPUmupmH-py3.9 in /home/tomasi/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (1.0s)

Writing lock file

Package operations: 26 installs, 0 updates, 0 removals

  • Installing certifi (2022.9.14)
  • Installing charset-normalizer (2.1.1)
  • Installing idna (3.4)
  • Installing markupsafe (2.1.1)
  • Installing pyparsing (3.0.9)
  • Installing pytz (2022.2.1)
  • Installing urllib3 (1.26.12)
  • Installing zipp (3.8.1)
  • Installing alabaster (0.7.12): Installing...
  • Installing alabaster (0.7.12)
  • Installing babel (2.10.3)
  • Installing docutils (0.19)
  • Installing imagesize (1.4.1)
  • Installing importlib-metadata (4.12.0)
  • Installing jinja2 (3.1.2)
  • Installing packaging (21.3)
  • Installing pygments (2.13.0)
  • Installing requests (2.28.1)
  • Installing snowballstemmer (2.2.0)
  • Installing sphinxcontrib-applehelp (1.0.2)
  • Installing sphinxcontrib-devhelp (1.0.2)
  • Installing sphinxcontrib-htmlhelp (2.0.0)
  • Installing sphinxcontrib-jsmath (1.0.1)
  • Installing sphinxcontrib-qthelp (1.0.3)
  • Installing sphinxcontrib-serializinghtml (1.1.5)
  • Installing sphinx (5.1.1)
  • Installing sphinxcontrib-asciinema (0.3.5): Failed

  CalledProcessError

  Command '['/home/tomasi/.cache/pypoetry/virtualenvs/test-asciinema-GPUmupmH-py3.9/bin/python', '/home/tomasi/miniconda3/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/home/tomasi/.cache/pypoetry/virtualenvs/test-asciinema-GPUmupmH-py3.9', '--no-deps', '/home/tomasi/.cache/pypoetry/artifacts/78/4c/9a/cb60406ce831b828cd20b97f4662f465ca0a0017e2891a75369c722596/sphinxcontrib.asciinema-0.3.5.tar.gz']' returned non-zero exit status 1.

  at ~/miniconda3/lib/python3.9/subprocess.py:528 in run
       524│             # We don't call process.wait() as .__exit__ does that for us.
       525│             raise
       526│         retcode = process.poll()
       527│         if check and retcode:
    →  528│             raise CalledProcessError(retcode, process.args,
       529│                                      output=stdout, stderr=stderr)
       530│     return CompletedProcess(process.args, retcode, stdout, stderr)
       531│ 
       532│ 

The following error occurred when trying to handle this error:

  EnvCommandError

  Command ['/home/tomasi/.cache/pypoetry/virtualenvs/test-asciinema-GPUmupmH-py3.9/bin/python', '/home/tomasi/miniconda3/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/home/tomasi/.cache/pypoetry/virtualenvs/test-asciinema-GPUmupmH-py3.9', '--no-deps', '/home/tomasi/.cache/pypoetry/artifacts/78/4c/9a/cb60406ce831b828cd20b97f4662f465ca0a0017e2891a75369c722596/sphinxcontrib.asciinema-0.3.5.tar.gz'] errored with the following return code 1, and output: 
  Processing /home/tomasi/.cache/pypoetry/artifacts/78/4c/9a/cb60406ce831b828cd20b97f4662f465ca0a0017e2891a75369c722596/sphinxcontrib.asciinema-0.3.5.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'error'
    error: subprocess-exited-with-error

    × Getting requirements to build wheel did not run successfully.
    │ exit code: 1
    ╰─> [17 lines of output]
        Traceback (most recent call last):
          File "/tmp/tmp3vna98g0_in_process.py", line 363, in <module>
            main()
          File "/tmp/tmp3vna98g0_in_process.py", line 345, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
          File "/tmp/tmp3vna98g0_in_process.py", line 130, in get_requires_for_build_wheel
            return hook(config_settings)
          File "/tmp/pip-build-env-tak_50cg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
            return self._get_build_requires(config_settings, requirements=['wheel'])
          File "/tmp/pip-build-env-tak_50cg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
            self.run_setup()
          File "/tmp/pip-build-env-tak_50cg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 482, in run_setup
            super(_BuildMetaLegacyBackend,
          File "/tmp/pip-build-env-tak_50cg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 335, in run_setup
            exec(code, locals())
          File "<string>", line 2, in <module>
        ImportError: cannot import name 'asciinema' from 'sphinxcontrib' (unknown location)
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

  at ~/miniconda3/lib/python3.9/site-packages/poetry/utils/env.py:1473 in _run
      1469│                 output = subprocess.check_output(
      1470│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1471│                 )
      1472│         except CalledProcessError as e:
    → 1473│             raise EnvCommandError(e, input=input_)
      1474│ 
      1475│         return decode(output)
      1476│ 
      1477│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:

  PoetryException

  Failed to install /home/tomasi/.cache/pypoetry/artifacts/78/4c/9a/cb60406ce831b828cd20b97f4662f465ca0a0017e2891a75369c722596/sphinxcontrib.asciinema-0.3.5.tar.gz

  at ~/miniconda3/lib/python3.9/site-packages/poetry/utils/pip.py:51 in pip_install
       47│ 
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│ 

Note that the long error message hints to a problem with pip, which can actually be reproduced by running the same command attempted by poetry:

# See the output of `poetry update` above: the following call to `pip` is taken from there
$ /home/tomasi/.cache/pypoetry/virtualenvs/test-asciinema-GPUmupmH-py3.9/bin/python /home/tomasi/miniconda3/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip install --use-pep517 --disable-pip-version-check --prefix /home/tomasi/.cache/pypoetry/virtualenvs/test-asciinema-GPUmupmH-py3.9 --no-deps /home/tomasi/.cache/pypoetry/artifacts/78/4c/9a/cb60406ce831b828cd20b97f4662f465ca0a0017e2891a75369c722596/sphinxcontrib.asciinema-0.3.5.tar.gz
Processing /home/tomasi/.cache/pypoetry/artifacts/78/4c/9a/cb60406ce831b828cd20b97f4662f465ca0a0017e2891a75369c722596/sphinxcontrib.asciinema-0.3.5.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/tmp/tmp1dhrnp1c_in_process.py", line 363, in <module>
          main()
        File "/tmp/tmp1dhrnp1c_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/tmp/tmp1dhrnp1c_in_process.py", line 130, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-vb_g0lem/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-vb_g0lem/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-vb_g0lem/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 482, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-vb_g0lem/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 2, in <module>
      ImportError: cannot import name 'asciinema' from 'sphinxcontrib' (unknown location)
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

This is the content of pyproject.toml as produced by poetry:

[tool.poetry]
name = "test-asciinema"
version = "0.1.0"
description = "test"
authors = ["Myself"]
license = "GPL2"
readme = "README.md"
packages = [{include = "test_asciinema"}]

[tool.poetry.dependencies]
python = "^3.9"
Sphinx = "^5.1.1"
"sphinxcontrib.asciinema" = "^0.3.5"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Here is some information about the system I am using:

$ python --version
Python 3.9.5
$ poetry --version
Poetry (version 1.2.0)

[kloczek]

I can reproduce this bug using poetry:

Please try to use generic procedure which is independent from pep517 backend by executing python3 -sBm build -w --no-isolation (--no-isolation is to relay only on locally installed modules without downloading .whl from pypi repo)

[ziotom78]

Hi @kloczek , thanks for your answer. Sorry, where should I run that command? I tried in the folder where I created the mock project, but it says

/home/tomasi/miniconda3/bin/python3: No module named build

[kloczek]

build module is available in pypi repo https://pypi.org/project/build/

[ziotom78]

Ok, I installed build and added a few empty files in the folder I created before to make it happy:

mkdir test_asciinema
touch test_asciinema/__init__.py
touch README.md

With these files, build completes successfully:

$ python3 -sBm build -w --no-isolation
* Getting dependencies for wheel...
* Building wheel...
Successfully built test_asciinema-0.1.0-py3-none-any.whl

Does this hint why poetry fails?

[kloczek]

Ok, I installed build and added a few empty files in the folder I created before to make it happy:

Di ypu made that on git repo content? 🤔

Does this hint why poetry fails?

No idea ..

[ziotom78]

Di ypu made that on git repo content? :thinking:

No, I did it on the mock example I created above with poetry init. It was an empty project built just for the purpose of replicating this issue.

Originally, I stumbled on this error on a much bigger codebase. I can try to use build on that too, but I will need some more time.

[zerocewl]

i can reproduce this too with poetry 1.2.x (which blocks all of our ci jobs)

any workarounds available (without dropping poetry 1.2)?

maybe some issues in the poetry repo can give a hint?

https://github.com/python-poetry/poetry/issues/6407 https://github.com/python-poetry/poetry/issues/6421

https://github.com/boisgera/pandoc/pull/49

https://peps.python.org/pep-0517/#summary-of-changes-to-pep-517

https://python-poetry.org/docs/pyproject#poetry-and-pep-517

[zerocewl]

@ziotom78 @divi255

i can reproduce the error with a plain pip install --use-pep517 sphinxcontrib.asciinema==0.3.5 --no-cache command.

The key is that a first install in a new and clean virtualenv passes, but after a manual pip uninstall sphinxcontrib.asciinema the same pip install ... cmd from above fails (but again only with the --use-pep517 flag).

The shell commands to reproduce:

python -m virtualenv testEnv
source testEnv/bin/activate
pip install --use-pep517 sphinxcontrib.asciinema==0.3.5 --no-cache  # this one passes without an error
pip uninstall sphinxcontrib.asciinema 
pip install --use-pep517 sphinxcontrib.asciinema==0.3.5 --no-cache # now the error is thrown
...
ImportError: cannot import name 'asciinema' from 'sphinxcontrib' (unknown location)
...

Edit1:

The install without --use-pep517 flag succeeds in the mentioned test virtualenv e.g. pip install sphinxcontrib.asciinema==0.3.5 --no-cache

Edit2:

If all of the sphinxcontrib-x packages are removed the next pep517 install call will pass again.

pip uninstall sphinxcontrib-applehelp sphinxcontrib-devhelp sphinxcontrib-qthelp sphinxcontrib-serializinghtml sphinxcontrib-jsmath sphinxcontrib-htmlhelp -y
pip install --use-pep517 sphinxcontrib.asciinema==0.3.5 --no-cache  # this one passes again

Edit3:

Added debug infos:

❯ python -V && pip list
Python 3.10.6
Package                       Version
----------------------------- ---------
alabaster                     0.7.12
Babel                         2.10.3
certifi                       2022.9.24
charset-normalizer            2.1.1
docutils                      0.19
idna                          3.4
imagesize                     1.4.1
Jinja2                        3.1.2
MarkupSafe                    2.1.1
packaging                     21.3
pip                           22.2.2
Pygments                      2.13.0
pyparsing                     3.0.9
pytz                          2022.2.1
requests                      2.28.1
setuptools                    65.4.0
snowballstemmer               2.2.0
Sphinx                        5.2.1
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
urllib3                       1.26.12
wheel                         0.37.1

[hoefling]

@divi255 looks to me like this issue is now resolved by #21 and can be closed; the error can't be reproduced with at least the latest 0.3.7 version.

[divi255]

great news. closing

[kloczek]

If may I ask next time to make release to have entry on https://github.com/divi255/sphinxcontrib.asciinema/releases github sends automatic notyficsyions ony about releases.

Thank you 👍

[divi255]

added into the internal makefile to do blank releases together with tags

[kloczek]

Thank you 👍