With removal of distutils in python3.12 from setuptools._distutils.dep_util import newer_group does not seem to work in pegen

[zokrezyl]

Bug report

Bug description:

# Add a code block here, if required

def compile_c_extension(
    generated_source_path: str,
    build_dir: Optional[str] = None,
    verbose: bool = False,
    keep_asserts: bool = True,
    disable_optimization: bool = False,
    library_dir: Optional[str] = None,
) -> pathlib.Path:
    """Compile the generated source for a parser generator into an extension module.

    The extension module will be generated in the same directory as the provided path
    for the generated source, with the same basename (in addition to extension module
    metadata). For example, for the source mydir/parser.c the generated extension
    in a darwin system with python 3.8 will be mydir/parser.cpython-38-darwin.so.

    If *build_dir* is provided, that path will be used as the temporary build directory
    of distutils (this is useful in case you want to use a temporary directory).

    If *library_dir* is provided, that path will be used as the directory for a
    static library of the common parser sources (this is useful in case you are
    creating multiple extensions).
    """
    import setuptools.command.build_ext
    import setuptools.logging
    from setuptools import Distribution, Extension
    from setuptools._distutils.ccompiler import new_compiler
    from setuptools._distutils.dep_util import newer_group
    from setuptools._distutils.sysconfig import customize_compiler

CPython versions tested on:

3.12 3.14

Operating systems tested on:

macOS

[picnixz]

(I don't really have a better label than build since it's about building something...). We should address it but I don't really know how. We could set some requirements on the Python version and setuptools, or we should modernize the function.

[zokrezyl]

Would be great to expose also the cpython generator in the pegen library. (including Tokens file, for experimenting with custom tokens) into https://github.com/we-like-parsers/pegen/blob/main/src/pegen/__main__.py. Maybe I should raise a ticket there.

I wish I could help, but am not familiar with the setuptools. I tried to install the standard-distutils tool, but that is throwing further errors. I lost now the context of my checkout, will add later some exceptions to this ticket.

[zokrezyl]

I played further, and looks like the remaining offending line is in setuptools/logging.py, line 38

  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/setuptools/logging.py", line 38, in set_threshold
    return set_threshold.unpatched(level)

I would just remove the '--compile-extension' feature. If someone wants to build extension, there are proper mechanism for it. At least for me it was a "trap"