setuptools==68.2.0 breaks colcon-python-setup-py

[hashb]

setuptools==68.2.0 breaks colcon-python-setup-py with the following:

Traceback (most recent call last):
  File "/<python_path>/site-packages/colcon_core/executor/__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "/<python_path>/site-packages/colcon_core/task/__init__.py", line 93, in __call__
    return await task_method(*args, **kwargs)
  File "/<python_path>/site-packages/colcon_ros/task/ament_python/build.py", line 51, in build
    setup_py_data = get_setup_data(self.context.pkg, env)
  File "/<python_path>/site-packages/colcon_core/task/python/__init__.py", line 20, in get_setup_data
    return dict(pkg.metadata[key](env))
  File "/<python_path>/site-packages/colcon_ros/package_augmentation/ros_ament_python.py", line 57, in getter
    return get_setup_information(
  File "/<python_path>/site-packages/colcon_python_setup_py/package_identification/python_setup_py.py", line 241, in get_setup_information
    _setup_information_cache[hashable_env] = _get_setup_information(
  File "/<python_path>/site-packages/colcon_python_setup_py/package_identification/python_setup_py.py", line 293, in _get_setup_information
    return ast.literal_eval(output)
  File "/<python_path>/ast.py", line 62, in literal_eval
    node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval')
  File "/<python_path>/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 1
    {'package_data': {}, 'dist_files': [], 'src_root': None, 'dependency_links': [], 'setup_requires': [], 'eager_resources': None, 'entry_points': {'pytest11': ['ament_lint = ament_lint.pytest_marker']}, 'exclude_package_data': None, 'extras_require': {}, 'include_package_data': None, 'install_requires': ['setuptools'], 'namespace_packages': None, 'packages': ['ament_lint'], 'python_requires': None, 'test_loader': None, 'test_runner': None, 'test_suite': None, 'tests_require': None, 'use_2to3': None, 'zip_safe': True, 'cffi_modules': None, 'verbose': 1, 'dry_run': 0, 'help': 0, 'command_packages': None, 'script_name': 'setup.py', 'script_args': ['--dry-run'], 'command_options': {}, 'package_dir': {}, 'py_modules': None, 'libraries': None, 'headers': None, 'ext_package': None, 'include_dirs': None, 'extra_path': None, 'scripts': None, 'data_files': [('share/ament_lint', ['package.xml']), ('share/ament_index/resource_index/packages', ['resource/ament_lint'])], 'password': '', 'command_obj': {}, 'have_run': {}, 'want_user_cfg': True, 'metadata': {'name': 'ament_lint', 'version': '0.12.7', 'author': 'Dirk Thomas', 'author_email': 'dthomas@osrfoundation.org', 'maintainer': 'Michael Jeronimo, Michel Hidalgo', 'maintainer_email': 'michael.jeronimo@openrobotics.org, michel@ekumenlabs.com', 'url': 'https://github.com/ament/ament_lint', 'license': 'Apache License, Version 2.0', 'description': 'Providing common API for ament linter packages.', 'long_description': 'Providing common API for ament linter packages, e.g. the `linter` marker for\npytest.', 'keywords': ['ROS'], 'platforms': None, 'classifiers': ['Intended Audience :: Developers', 'License :: OSI Approved :: Apache Software License', 'Programming Language :: Python', 'Topic :: Software Development'], 'download_url': 'https://github.com/ament/ament_lint/releases', 'provides': None, 'requires': None, 'obsoletes': None, 'long_description_content_type': None, 'project_urls': {}, 'license_file': None, '_normalized_install_requires': {'setuptools': <Requirement('setuptools')>}, '_normalized_extras_require': {}}}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           ^
SyntaxError: invalid syntax

Pinning setuptools==68.1.2 fixed this issue temporarily but doesn't fix the root cause.

There is a discussion on colon-core about setup.py deprecation https://github.com/colcon/colcon-core/issues/454 but doesn't look like its going to happen soon

[cottsay]

Is this a 100% hit rate with setuptools>=68.2.0 or intermittent?

[asasine]

It's 100% hit rate with setuptools==68.2.0. Upgrading to 68.2.1 mitigates the issue.

[cottsay]

Upgrading to 68.2.1 mitigates the issue.

That makes me think that setuptools is treating this like a bug in their code. If that's the case, I'm inclined to let this play out without making any changes to colcon.

Any other suggestions?

[asasine]

Are there backwards-compatible changes that can be made to this package that avoids invoking setup.py as a CLI, as this is a deprecated feature?

[carlosperez-slamcore]

Any updates on this?

[cottsay]

Any updates on this?

The solution is to use a different version of setuptools. The bug only exists in 68.2.0 - anything newer or older shouldn't have this bug.

The intermediate issue has to do with the way we're extracting information from setuptools. There are PRs open on this repository with different approaches to that, but I'm not convinced that they would have avoided the 68.2.0 setuptools bug either.