test_simple_fstring and test_fstring_with_args fail with Python 3.12 beta

[tjanez]

Fedora is rebuilding all Python packages for the upcoming Python 3.12 upgrade in Fedora 39.

Pydocstyle's test suite is run as part of the RPM build process and there are 2 failures (which were not present before):

============================= test session starts ==============================
platform linux -- Python 3.12.0b3, pytest-7.3.2, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /builddir/build/BUILD/pydocstyle-6.3.0
configfile: pyproject.toml
collecting ... collected 214 items
src/tests/parser_test.py::test_function PASSED                           [  0%]

... trimmed ...

src/tests/test_utils.py::test_strip_non_alphanumeric PASSED              [100%]
=================================== FAILURES ===================================
_____________________________ test_simple_fstring ______________________________
    def test_simple_fstring():
        """Test parsing of a function with a simple fstring as a docstring."""
        parser = Parser()
        code = CodeSnippet("""\
            def do_something(pos_param0, pos_param1, kw_param0="default"):
                f\"""Do something.\"""
                return None
        """)
        module = parser.parse(code, 'file_path')
        assert module.is_public
        assert module.dunder_all is None

        function, = module.children
        assert function.name == 'do_something'
        assert function.decorators == []
        assert function.children == []
>       assert function.docstring == 'f"""Do something."""'
E       assert None == 'f"""Do something."""'
E        +  where None = Function(name='do_something', _source=['def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do something."""\n', '    return None\n'], start=1, end=3, decorators=[], docstring=None, children=[], callable_args=['pos_param0', 'pos_param1', 'kw_param0'], parent=Module(name='file_path', _source=['def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do something."""\n', '    return None\n'], start=1, end=4, decorators=[], docstring=None, children=[Function(name='do_something', _source=['def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do something."""\n', '    return None\n'], start=1, end=3, decorators=[], docstring=None, children=[], callable_args=['pos_param0', 'pos_param1', 'kw_param0'], parent=Module(name='file_path', _source=['def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do something."""\n', '    return None\n'], start=1, end=4, decorators=[], docstring=None, children=[...], parent=None, _dunder_all=None, dunder_all_error=None, future_imports=set(), skipped_error_codes=''), skipped_error_codes='')], parent=None, _dunder_all=None, dunder_all_error=None, future_imports=set(), skipped_error_codes=''), skipped_error_codes='').docstring
src/tests/parser_test.py:69: AssertionError
____________________________ test_fstring_with_args ____________________________
    def test_fstring_with_args():
        """Test parsing of a function with an fstring with args as a docstring."""
        parser = Parser()
        code = CodeSnippet("""\
            foo = "bar"
            bar = "baz"
            def do_something(pos_param0, pos_param1, kw_param0="default"):
                f\"""Do some {foo} and some {bar}.\"""
                return None
        """)
        module = parser.parse(code, 'file_path')
        assert module.is_public
        assert module.dunder_all is None

        function, = module.children
        assert function.name == 'do_something'
        assert function.decorators == []
        assert function.children == []
>       assert function.docstring == 'f"""Do some {foo} and some {bar}."""'
E       assert None == 'f"""Do some {foo} and some {bar}."""'
E        +  where None = Function(name='do_something', _source=['foo = "bar"\n', 'bar = "baz"\n', 'def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do some {foo} and some {bar}."""\n', '    return None\n'], start=3, end=5, decorators=[], docstring=None, children=[], callable_args=['pos_param0', 'pos_param1', 'kw_param0'], parent=Module(name='file_path', _source=['foo = "bar"\n', 'bar = "baz"\n', 'def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do some {foo} and some {bar}."""\n', '    return None\n'], start=1, end=6, decorators=[], docstring=None, children=[Function(name='do_something', _source=['foo = "bar"\n', 'bar = "baz"\n', 'def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do some {foo} and some {bar}."""\n', '    return None\n'], start=3, end=5, decorators=[], docstring=None, children=[], callable_args=['pos_param0', 'pos_param1', 'kw_param0'], parent=Module(name='file_path', _source=['foo = "bar"\n', 'bar = "baz"\n', 'def do_something(pos_param0, pos_param1, kw_param0="default"):\n', '    f"""Do some {foo} and some {bar}."""\n', '    return None\n'], start=1, end=6, decorators=[], docstring=None, children=[...], parent=None, _dunder_all=None, dunder_all_error=None, future_imports=set(), skipped_error_codes=''), skipped_error_codes='')], parent=None, _dunder_all=None, dunder_all_error=None, future_imports=set(), skipped_error_codes=''), skipped_error_codes='').docstring
src/tests/parser_test.py:100: AssertionError
======================== 2 failed, 212 passed in 18.93s ========================

This is Fedora's build task in Koji.

Full RPM build log

[sigmavirus24]

This is because Py3.12 changed how strings are tokenized and broke most things. I suspect there hasn't been enough time for this to be updated. It's been tricky to do in other projects I've seen attempt it as well

[mgorny]

Ping. 3.12.0 is just around the corner, and this is now blocking other packages that are otherwise ready for 3.12 already.