Support for Python 3.11

[etseidler]

Are there any plans to support Python 3.11?

I ran my app's test suite after bumping Python from 3.9.7 to 3.11.0 and got a bunch of the following:

pony.orm.decompiling.DecompileError: Unsupported operation: RETURN_GENERATOR

[St4rG00se]

Hi,

I'm using Mutmut project for my mutation tests. This project uses Pony and since I upgraded from python 3.10.1 to python 3.11.0 I have the same issue

pony.orm.decompiling.DecompileError: Unsupported operation: RETURN_GENERATOR

to reproduce it with mutmut, just run mutation testing in order to get a .mutmut-cache (SQLite file) and run mutmut html (without error you should get an html report)

Best regards

[boxed]

Is there are full traceback available?

[St4rG00se]

In my case (with mutmut)

$ mutmut html

Traceback (most recent call last):
  File "/Users/.../bin/mutmut", line 8, in <module>
    sys.exit(climain())
             ^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/mutmut/__init__.py", line 882, in wrapper
    f(*args, **kwargs)
  File "/Users/.../lib/python3.11/site-packages/mutmut/__main__.py", line 234, in html
    create_html_report(dict_synonyms)
  File "/Users/.../lib/python3.11/site-packages/mutmut/cache.py", line 88, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in create_html_report
  File "/Users/.../lib/python3.11/site-packages/pony/orm/core.py", line 519, in new_func
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/mutmut/cache.py", line 264, in create_html_report
    mutants = list(select(x for x in Mutant))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/pony/orm/core.py", line 5560, in select
    return make_query(args, frame_depth=cut_traceback_depth+1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/pony/orm/core.py", line 5546, in make_query
    tree, external_names, cells = decompile(gen)
                                  ^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/pony/orm/decompiling.py", line 38, in decompile
    decompiler = Decompiler(codeobject)
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../lib/python3.11/site-packages/pony/orm/decompiling.py", line 156, in __init__
    decompiler.decompile()
  File "/Users/.../lib/python3.11/site-packages/pony/orm/decompiling.py", line 256, in decompile
    throw(DecompileError('Unsupported operation: %s' % opname))
  File "/Users/.../lib/python3.11/site-packages/pony/utils/utils.py", line 99, in throw
    raise exc
pony.orm.decompiling.DecompileError: Unsupported operation: RETURN_GENERATOR

[boxed]

https://docs.python.org/3/library/dis.html#opcode-RETURN_GENERATOR seems to be the new addition to the python bytecode.

https://github.com/python/cpython/issues/90567 for the issue with more information.

[baldurmen]

Here's a complete tracelog of the testsuite failure running 3.11, it can help.

pony_3.11_failures.txt

FAILED (failures=4, errors=3348, skipped=1)

[sashaaero]

Hello this is being in work. You can check it here: https://github.com/ponyorm/pony/pull/671

[j4hangir]

Hello this is being in work. You can check it here: #671

I installed it with pip install git+https://github.com/jspricke/pony@py311, still get pony.orm.decompiling.DecompileError: Unsupported operation: RESUME when performing .order_by(lambda m: m.created)]

[0xWTC]

I can confirm lambdas don't work under Python 3.11 on the latest Pony version

[magwas]

Ran into the same while trying to use mutmut.

[0xalpharush]

+1

[lucabravi]

I can confirm lambdas don't work under Python 3.11 on the latest Pony version

+1

[Nimja]

+1

[daoye]

same problem.

python 3.11.3 pony 0.7.16 flask 2.3.2 mariadb 10.3.32

`DecompileError

pony.orm.decompiling.DecompileError: Unsupported operation: RETURN_GENERATOR Traceback (most recent call last)

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/flask/app.py", line 2213, in __call__

return self.wsgi_app(environ, start_response)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/flask/app.py", line 2193, in wsgi_app

response = self.handle_exception(e)
           ^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app

response = self.full_dispatch_request()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request

rv = self.handle_user_exception(e)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request

rv = self.dispatch_request()
     ^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request

return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/projects/mhong/mhong-console/app/controllers/resource.py", line 27, in list

f = select(o for o in Item)
    ^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/pony/orm/core.py", line 5560, in select

return make_query(args, frame_depth=cut_traceback_depth+1)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/pony/orm/core.py", line 5546, in make_query

tree, external_names, cells = decompile(gen)
                              ^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/pony/orm/decompiling.py", line 38, in decompile

decompiler = Decompiler(codeobject)
             ^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/pony/orm/decompiling.py", line 156, in __init__

decompiler.decompile()
^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/pony/orm/decompiling.py", line 256, in decompile

throw(DecompileError('Unsupported operation: %s' % opname))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/april/.virtualenvs/mh_console/lib/python3.11/site-packages/pony/utils/utils.py", line 99, in throw

raise exc
^^^^^^^^^

pony.orm.decompiling.DecompileError: Unsupported operation: RETURN_GENERATOR

The debugger caught an exception in your WSGI application. You can now look at the traceback which led to the error.

To switch between the interactive traceback and the plaintext one, you can click on the "Traceback" headline. From the text traceback you can also create a paste of it. For code execution mouse-over the frame you want to debug and click on the console icon on the right side.

You can execute arbitrary Python code in the stack frames and there are some extra helpers available for introspection:

dump() shows all variables in the frame
dump(obj) dumps all that's known about the object

`

[esingtse]

+1

[lw4z]

+1

[aladopoulos]

+1 same problem while trying to use mutmut

[devarops]

➕1️⃣ mutmut issue

[timini]

+1 trying to use mutmut here

[nnsdtr]

+1

[ne-bknn]

Came from trying to use mutmut on 3.11 on arch, installing python-pony from repo instead of pypi fixed the issue.

[fluffy-critter]

I'm still having trouble with Pony on 3.11 where lambdas are being used; it seems to be the same issue reported in #681, which shouldn't have been closed IMO since "regressing to Python 3.10" is not an adequate long-term fix.

Is there any attention on that particular issue? More and more of my Python environments are defaulting to 3.11 and it's getting a little painful to manage.

[edmiester777]

looks to have a PR open #671 for anyone looking for this. I've just installed from that branch in the meantime, though you must accept some level of risk as tests were failing. Seems good to me.

[fluffy-critter]

It looks like #671 has been successfully merged. Is there any plan to do a pypi release? The current pypi is pretty outdated anyway (January 2022 at present).

[kozlovsky]

The official PonyORM 0.7.17 release with Python 3.11 support is out! In addition to fixes of #671 PR, it also fixes LIST_APPEND, LIST_TO_TUPLE, and EXTENDED_ARG opcodes

[fluffy-critter]

Awesome! Thank you so much for all your hard work.