search

flyingcircusio / batou

batou is a universal, fractal deployment utility using Python.
https://batou.readthedocs.org
Other
47 stars 12 forks source link

Too old setuptools in supervisor #366

Closed gforcada closed 1 month ago

gforcada commented 1 year ago

When trying to deploy with batou a component that requires anything above python 3.7 a new venv to install zc.buildout and supervisor is created but that fails:

lib2to3 was removed from setuptools in version 58.0.0 but batou tries to create a venv with setuptools 50.3.2, see the code.

Seems that this is somewhat related to being python 3.7 or 3.8 available in the target system. Starting with 3.9 supervisor's venv fails to get installed as mentioned above.

Our current trick is that whenever we have multiple python versions, we create a Nix runtime with:

...
in buildEnv {
    paths = [
      (runCommand "profile" { } "install -D ${shellInit} $out/etc/profile.d/runtime.sh")
      python39
      (python37.overrideAttrs(_: { meta.priority = 0; }))
...

This way, python3.7 is picked as the default python 3 version. Is that the right approach? Can we bump setuptools version?

See the full traceback below:

/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/dist.py:667: SetuptoolsDeprecationWarning: As setuptools moves its configuration towards `pyproject.toml`,
`setuptools.config.parse_configuration` became deprecated.

For the time being, you can use the `setuptools.config.setupcfg` module
to access a backward compatible API, but this module is provisional
and might be removed in the future.

  parse_configuration(self, self.command_options,
WARNING: The easy_install command is deprecated and will be removed in a future version.
/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/dist.py:667: SetuptoolsDeprecationWarning: As setuptools moves its configuration towards `pyproject.toml`,
`setuptools.config.parse_configuration` became deprecated.

For the time being, you can use the `setuptools.config.setupcfg` module
to access a backward compatible API, but this module is provisional
and might be removed in the future.

  parse_configuration(self, self.command_options,
Traceback (most recent call last):
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 152, in save_modules
    yield saved
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 193, in setup_context
    yield
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 254, in run_setup
    _execfile(setup_script, ns)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 43, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-sn4im3og/zc.recipe.egg-2.0.7/setup.py", line 26, in <module>
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/bdist_egg.py", line 167, in run
    cmd = self.call_command('install_lib', warn_dir=0)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/bdist_egg.py", line 153, in call_command
    self.run_command(cmdname)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/command/install_lib.py", line 105, in build
    self.run_command('build_py')
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/build_py.py", line 55, in run
    self.build_package_data()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/build_py.py", line 126, in build_package_data
    srcfile in self.distribution.convert_2to3_doctests):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Distribution' object has no attribute 'convert_2to3_doctests'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/easy_install.py", line 2286, in main
    setup(
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/easy_install.py", line 408, in run
    self.easy_install(spec, not self.no_deps)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/easy_install.py", line 650, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/easy_install.py", line 695, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/easy_install.py", line 880, in install_eggs
    return self.build_and_install(setup_script, setup_base)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/easy_install.py", line 1150, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/easy_install.py", line 1134, in run_setup
    run_setup(setup_script, args)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 244, in run_setup
    with setup_context(setup_dir):
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 185, in setup_context
    with save_modules():
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 164, in save_modules
    saved_exc.resume()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 139, in resume
    raise exc.with_traceback(self._tb)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 152, in save_modules
    yield saved
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 193, in setup_context
    yield
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 254, in run_setup
    _execfile(setup_script, ns)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/sandbox.py", line 43, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-sn4im3og/zc.recipe.egg-2.0.7/setup.py", line 26, in <module>
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/bdist_egg.py", line 167, in run
    cmd = self.call_command('install_lib', warn_dir=0)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/bdist_egg.py", line 153, in call_command
    self.run_command(cmdname)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/command/install_lib.py", line 105, in build
    self.run_command('build_py')
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/srv/s-derfreitag/.nix-profile/lib/python3.11/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/build_py.py", line 55, in run
    self.build_package_data()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/setuptools/command/build_py.py", line 126, in build_package_data
    srcfile in self.distribution.convert_2to3_doctests):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Distribution' object has no attribute 'convert_2to3_doctests'
While:
  Installing.
  Getting section supervisor.
  Initializing section supervisor.
  Installing recipe zc.recipe.egg.
  Getting distribution for 'zc.recipe.egg==2.0.7'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/buildout.py", line 2174, in main
    getattr(buildout, command)(args)
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/buildout.py", line 701, in install
    [self[part]['recipe'] for part in install_parts]
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/buildout.py", line 701, in <listcomp>
    [self[part]['recipe'] for part in install_parts]
     ~~~~^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/buildout.py", line 1324, in __getitem__
    options._initialize()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/buildout.py", line 1432, in _initialize
    self.initialize()
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/buildout.py", line 1438, in initialize
    recipe_class = _install_and_load(reqs, 'zc.buildout', entry, buildout)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/buildout.py", line 1381, in _install_and_load
    zc.buildout.easy_install.install(
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/easy_install.py", line 957, in install
    return installer.install(specs, working_set)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/easy_install.py", line 682, in install
    for dist in self._get_dist(requirement, ws):
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/easy_install.py", line 574, in _get_dist
    dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/s-derfreitag/deployment/work/supervisor/lib/python3.11/site-packages/zc/buildout/easy_install.py", line 1745, in _move_to_eggs_dir_and_compile
    [tmp_loc] = glob.glob(os.path.join(tmp_dest, '*'))
    ^^^^^^^^^
ValueError: not enough values to unpack (expected 1, got 0)
elikoga commented 1 year ago