Importing `glom` on Python3.12 fails

[joerick]

Didn't happen when I was using cProfile. Here is the stacktrace: glom version 23.5.0 pyinstrument version 4.6.2 python version 3.12.2 TypeError Traceback (most recent call last) File Untitled-1:5 3 p = pyinstrument.Profiler() 4 p.start() ----> 5 import glom 6 p.stop()

File venv/lib/python3.12/site-packages/glom/init.py:48 34 from glom.reduction import Sum, Fold, Flatten, flatten, FoldError, Merge, merge 35 from glom.matching import (M, 36 Or, 37 And, (...) 46 Check, 47 CheckError) ---> 48 from glom.mutation import Assign, Delete, assign, delete, PathDeleteError 50 # there's no -ion word that really fits what "streaming" means. 51 # generation, production, iteration, all have more relevant meanings 52 # elsewhere. (maybe procrastination :P) 53 from glom.streaming import Iter

File .venv/lib/python3.12/site-packages/glom/mutation.py:209 204 return glom(obj, Assign(path, val, missing=missing)) 207 _ALL_BUILTIN_TYPES = [v for v in builtins.values() if isinstance(v, type)] 208 _BUILTIN_BASE_TYPES = [v for v in _ALL_BUILTIN_TYPES --> 209 if not issubclass(v, tuple([t for t in _ALL_BUILTIN_TYPES 210 if t not in (v, type, object)]))] 211 _UNASSIGNABLE_BASE_TYPES = tuple(set(_BUILTIN_BASE_TYPES) 212 - set([dict, list, BaseException, object, type])) 215 def _set_sequence_item(target, idx, val):

TypeError: issubclass() arg 2 must be a class, a tuple of classes, or a union

Originally posted by @AvivSamet-Silk in https://github.com/joerick/pyinstrument/issues/288#issuecomment-2050154070

[joerick]

I can only reproduce on python 3.12. It doesn't occur on 3.11 or earlier. This might be a Python bug, it's very strange.

Here's a sample readout (python 3.12.4, pyinstrument 4.7.2). Reducing the interval makes the crash easier to reproduce.

$ env312/bin/pyinstrument -i 0.0001  test.py
/Users/joerick/Desktop/glom/glom/core.py:506: RuntimeWarning: assigning None to unbound local 'k'
  _BUILTIN_ID_NAME_MAP = dict([(id(v), k)
/Users/joerick/Desktop/glom/glom/core.py:506: RuntimeWarning: assigning None to unbound local 'v'
  _BUILTIN_ID_NAME_MAP = dict([(id(v), k)
/Users/joerick/Desktop/glom/glom/mutation.py:211: RuntimeWarning: assigning None to unbound local 'v'
  if not issubclass(v, tuple([t for t in _ALL_BUILTIN_TYPES
Traceback (most recent call last):
  File "/Users/joerick/Desktop/glom/glom/core.py", line 2158, in register_op
    handler = auto_func(t)
              ^^^^^^^^^^^^
  File "/Users/joerick/Desktop/glom/glom/mutation.py", line 236, in _assign_autodiscover
    if issubclass(type_obj, _UNASSIGNABLE_BASE_TYPES):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: issubclass() arg 2 must be a class, a tuple of classes, or a union

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/joerick/Desktop/glom/env312/bin/pyinstrument", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/joerick/Desktop/glom/env312/lib/python3.12/site-packages/pyinstrument/__main__.py", line 382, in main
    exec(code, globs, None)
  File "<string>", line 1, in <module>
  File "<frozen runpy>", line 286, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "test.py", line 1, in <module>
    import glom
  File "/Users/joerick/Desktop/glom/glom/__init__.py", line 48, in <module>
    from glom.mutation import Assign, Delete, assign, delete, PathDeleteError
  File "/Users/joerick/Desktop/glom/glom/mutation.py", line 247, in <module>
    register_op('assign', auto_func=_assign_autodiscover, exact=False)
  File "/Users/joerick/Desktop/glom/glom/core.py", line 2437, in register_op
    _DEFAULT_SCOPE[TargetRegistry].register_op(op_name, **kwargs)
  File "/Users/joerick/Desktop/glom/glom/core.py", line 2160, in register_op
    raise TypeError('error while determining support for operation'
TypeError: error while determining support for operation "assign" on target type: OrderedDict (got TypeError('issubclass() arg 2 must be a class, a tuple of classes, or a union'))

The warnings and crashes appear around these lines:

https://github.com/mahmoud/glom/blob/3cd57b6a6e04d522ddbab6869a040c408bff8b7a/glom/core.py#L500-L507

https://github.com/mahmoud/glom/blob/3cd57b6a6e04d522ddbab6869a040c408bff8b7a/glom/mutation.py#L207-L212

And indeed it doesn't crash with cProfile or profile. But they do crash with pyinstrument, all the way back to v2.0

Something else caught my eye in one of these outputs. We get these errors 'Assigning None to unbound local x'. I once saw this:

/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/opcode.py:492: RuntimeWarning: assigning None to unbound local 'opcode'

That code was actually in Python itself!

_inline_cache_entries = [
    sum(_cache_format.get(opname[opcode], {}).values()) for opcode in range(256)
]

The thing that has in common with the others is that this is a list comprehension at the module level.

[joerick]

This issue from cpython appears relevant. I think the root cause is us calling LocalsToFast with the clear parameter. setting the clear parameters to 0 seems to solve the issue. But now I want to figure out if we need to call FastToLocals and LocalsToFast at all. We're not planning to change the locals anyway.