Test suite fails with Python 3.10a3

[FelixSchwarz]

When switching from Python 3.10a2 -> 3.10a3 the test suite starts failing.

======================================================================
FAIL: test_not_iterable (genshi.template.tests.directives.ForDirectiveTestCase)
Verify that assignment to nested tuples works correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "…/genshi/template/tests/directives.py", line 489, in test_not_iterable
    list(tmpl.generate(foo=12))
TypeError: 'int' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "…/genshi/template/tests/directives.py", line 505, in test_not_iterable
    self.assertEqual(2, frames[-1].tb_lineno)
AssertionError: 2 != 61

======================================================================
FAIL: test_error_access_undefined (genshi.template.tests.eval.ExpressionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "…/genshi/template/tests/eval.py", line 444, in test_error_access_undefined
    expr.evaluate({})
genshi.template.eval.UndefinedError: "nothing" not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "…/genshi/template/tests/eval.py", line 458, in test_error_access_undefined
    self.assertEqual(50, frames[-3].tb_lineno)
AssertionError: 50 != 109

======================================================================
FAIL: test_error_getattr_undefined (genshi.template.tests.eval.ExpressionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "…/genshi/template/eval.py", line 301, in lookup_attr
    val = getattr(obj, key)
AttributeError: 'Something' object has no attribute 'nil'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "…/genshi/template/eval.py", line 307, in lookup_attr
    val = obj[key]
TypeError: 'Something' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "…/genshi/template/tests/eval.py", line 467, in test_error_getattr_undefined
    expr.evaluate({'something': Something()})
genshi.template.eval.UndefinedError: <Something> has no member named "nil"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "…/genshi/template/tests/eval.py", line 482, in test_error_getattr_undefined
    self.assertEqual(50, frame.tb_lineno)
AssertionError: 50 != 109

======================================================================
FAIL: test_error_getitem_undefined_string (genshi.template.tests.eval.ExpressionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "…/genshi/template/eval.py", line 318, in lookup_item
    return obj[key]
TypeError: 'Something' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "…/genshi/template/tests/eval.py", line 491, in test_error_getitem_undefined_string
    expr.evaluate({'something': Something()})
genshi.template.eval.UndefinedError: <Something> has no member named "nil"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "…/genshi/template/tests/eval.py", line 506, in test_error_getitem_undefined_string
    self.assertEqual(50, frame.tb_lineno)
AssertionError: 50 != 109

----------------------------------------------------------------------
Ran 886 tests in 0.547s

FAILED (failures=4)

Initial report: https://bugzilla.redhat.com/show_bug.cgi?id=1907400

[FelixSchwarz]

Similar failures with Python 3.10rc2 with one minor difference:

…
  File "…/genshi/template/tests/eval.py", line 506, in test_error_getitem_undefined_string
    self.assertEqual(50, frame.tb_lineno)
AssertionError: 50 != -1

Differences in line numbers are likely caused by PEP 626.

[hodgestar]

How did a line number end up being -1?

[hodgestar]

After digging a bit, I'm pretty sure that the arguments for CodeType changed again as a result of PEP 626. Admittedly these arguments are undocumented, so we can't complain too much -- https://docs.python.org/3/library/types.html#types.CodeType.

There are two functions in genshi.compat that are affected -- get_code_params and build_code_chunk.

For build_code_chunk we can probably write a Python >= 3.8 version that calls the new .replace(...) method on the code object to set the line number.

For get_code_params I'm not quite sure what to do yet -- probably we need to go look at the 3.10 source code and see what has happened to CodeType and update the values returned.

[FelixSchwarz]

For build_code_chunk we can probably write a Python >= 3.8 version that calls the new .replace(...) method on the code object to set the line number.

Thank you for the hint. I did that in #49 and all errors just vanished :-)

However...

For get_code_params I'm not quite sure what to do yet -- probably we need to go look at the 3.10 source code and see what has happened to CodeType and update the values returned.

I guess this means get_code_params() is never really executed by the test suite... (?)

[FelixSchwarz]

Actually we got a hint from the BFDL a while ago: https://github.com/edgewall/genshi/issues/43#issuecomment-845491080

[hodgestar]

get_code_params() is only used when a genshi.template.eval.Code object is serialized. This is actually tested (look for pickle in genshi.template.tests.eval) but nothing checks whether the code object is correct afterwards.