Closed juliangilbey closed 10 months ago
nicoddemus
commented
10 months ago Hi,
What is happening is that the w = MyWidget() instance is leaking to test_2-- this used to happen because we would keep a reference to the error traceback, which would contain the locals of test_1, but that really should not be the case because we clear the exceptions between tests now:
Not sure what else could be causing this in your system, I'm afraid.
juliangilbey
commented
10 months ago Thanks @nicoddemus! I couild try tracing that function perhaps? It's clearly some difference in behaviour between Python 3.11 and Python 3.12, at least on my chroot. Would you be able to add Python 3.12 to the list of Python versions tested in .github/workflows/main.yml (and perhaps tox.ini too, which doesn't yet include 3.11, BTW)? That way, we would be able to see if it's my system or whether it's a more consistent problem.
juliangilbey
commented
10 months ago It seems like the Python 3.12 behaviour is quite different from the Python 3.11 behaviour. I don't understand enough about PyQt5 and Python 3.12 vs Python 3.11 to know what has caused this. Here's what I tried. I modified the test to read:
def test_1(qapp):
global weak_ref
w = MyWidget()
weak_ref = weakref.ref(w)
print("weak_ref in test_1 =", weak_ref())
qapp.postEvent(w, qt_api.QtCore.QEvent(qt_api.QtCore.QEvent.Type.User))
qapp.processEvents()
def test_2(qapp):
assert called
print("weak_ref in test_2 pre-collect =", weak_ref())
gc.collect()
print("weak_ref in test_2 post-collect =", weak_ref())
assert weak_ref() is None
print("weak_ref in test_2 post-assert =", weak_ref())
"""
)
result = testdir.runpytest("--capture=tee-sys")
result.stdout.fnmatch_lines(["*1 failed, 1 passed*"])This allows me to see how the weak_ref object changes over time. With Python 3.11, I get the following (dropping everything other than the print statement outputs):
weak_ref in test_1 = <test_exceptions_dont_leak.MyWidget object at 0x7f160cec6950>
weak_ref in test_2 pre-collect = None
weak_ref in test_2 post-collect = None
weak_ref in test_2 post-assert = NoneBut with Python 3.12, I get:
weak_ref in test_1 = <test_exceptions_dont_leak.MyWidget object at 0x7f2e7d06f380>
weak_ref in test_2 pre-collect = <test_exceptions_dont_leak.MyWidget object at 0x7f2e7d06f380>
weak_ref in test_2 post-collect = <test_exceptions_dont_leak.MyWidget object at 0x7f2e7d06f380>So it seems that the object is not garbage-collected at all. This may be a result of the modified garbage collection behaviour in Python 3.12, but it seems a little bizarre for this to be the case. Perhaps something else is going on?
I then tried mimicking the pytest-qt script, but without any calls to qapp or pytest:
import gc
import weakref
class MyClass:
def __init__(self):
self.var = 1
weak_ref = None
def func1():
global weak_ref
c = MyClass()
weak_ref = weakref.ref(c)
print("weak_ref in func1 =", weak_ref())
def func2():
print("weak_ref in func2 pre-collect =", weak_ref())
gc.collect()
print("weak_ref in func2 post-collect =", weak_ref())
func1()
func2()and the results are as expected:
weak_ref in func1 = <__main__.MyClass object at 0x7fb5b4101990>
weak_ref in func2 pre-collect = None
weak_ref in func2 post-collect = Nonewith both Python 3.11 and Python 3.12.
I then modified it to use pytest:
def test_func1():
global weak_ref
c = MyClass()
weak_ref = weakref.ref(c)
assert weak_ref() is not None
def test_func2():
gc.collect()
assert weak_ref() is Noneand pytest succeeds with both Python 3.11 and Python 3.12.
I'm not sure what to suggest from here :(
nicoddemus
commented
10 months ago Thanks @juliangilbey for the detailed post.
I did not realize that we were not testing with Python 3.12 yet.
I added testing for CI in 3.12 and the problem exists in all platforms, which will help debug this greatly.
Regardless, I just released 4.3.1 which skips this test on Python 3.12, which should at least alleviate your immediate problem.
I plan to investigate this in more detail when I have some time.
Thanks again.
graingert
commented
10 months ago I ran this test and inspected the live object with objgraph:
https://docs.python.org/3/library/sys.html#sys.last_exc is new in 3.12
nicoddemus
commented
10 months ago Awesome @graingert, nailed that right on the head. 😁
juliangilbey
commented
10 months ago Oh wow, thanks @graingert; what great detective work!
Hi! Debian is in the process of migrating to Python 3.12, and we have discovered that one of the tests in pytest-qt fails with Python 3.12. Here is the output of running pytest using Python 3.12 and then Python 3.11. There are lots of warnings, but one failure in the 3.12 tests which passes in the 3.11 tests. These tests are being run in a clean chroot, and the version
qt-4.2.0+repackis just renamed from the originalqt-4.2.0(the contents are identical).