Dependency on debug version of Python

[andy-maier]

The current dependency on a debug version of Python is not unproblematic. For example, on MacOS there is no homebrew-installable debug version of Python. On Travis, a virtual env is already set up for the test code, but of course not with a debug version. On Appveyor, the virtualenv is not set up, but a Python version is already installed, of course again not a debug version. Probably the list goes on.

So I think it is worthwhile talking about how to support pytest-leaks with a normal (non-debug) version of Python.

I have started some work on basic plumming for that in https://github.com/andy-maier/python-yagot, where its GarbageTracker class could be used. Let me know what you think about this.

[seberg]

I am not sure that would be generally useful, i.e. dangerous to use as a default. Such a hack seems like it can work for programs written in Python, but for me this is super important to debug extension modules written in C, where the leaks are objects that do not implement cyclic garbage collection.

[pv]

I also suspect the refcount tracking probably cannot be "emulated" on non-debug Python builds.

In principle, pytest-leaks can (and does) track multiple types of leaks, so there could be room for extensions. However, since the gc collectable garbage is not error per se, there would also be a need for a mechanism to select what pytest-leaks should complain against (which probably should still error out if the user asks for stuff that's not available on non-debug build). The gc garbage tracking probably also conflicts with refcount tracking, so it may not be possible to do both at the same time.

[andy-maier]

@pv I also don't think that the sys.gettotalrefcount() function used by pytest-leaks can be emulated on non-debug builds.

I can see how using sys.gettotalrefcount() is a good and simple way to detect missing Py_DECREF() calls in C extension modules, but I'm not sure yet whether the gc.garbage based approach I used can do that as well. I would assume that a missed Py_DECREF() call would cause the referenced object to remain in gc.garbage even when the gc has run a full collection 3 times.

Anyway, I am not suggesting to replace the sys.gettotalrefcount() approach, but to add an approach that does not depend on the debug Python build.

[seberg]

@andy-maier from a Python side perspective that may even be true. But most basic python types are not garbage collected (except containers of course). E.g. strings are not, so if we construct a string for an error message, and forget the Py_DECREF there is no chance of detecting that.

Personally, I have a bit more doubt about the gc.garbage approach, as annoying a debug build can be... pytest-leaks is already pretty slow, and I would bet a gc.garbage approach is even slower. My gut feeling is that the garbage approach may be good to write individual tests, but not as a generic test runner.

[pv]

I expect that mis-refcounted objects will simply look like alive objects to Python gc. It can only know whether a given PyObject* is referenced from some C struct via (correct) refcounting.

[andy-maier]

@seberg Ok, I see that looking at uncollectable objects (those that stay in the garbage collector) cannot be detected that way. It is also true that using the gc.garbage approach on an entire testsuite is slow. In the pywbem project, using it on the entire set of >13k unit tests changes their run time from 4 minutes to 4 hours. Based on what we learned from such a long run, we have started writing individual tests that now run all the time (and they are quick).

@pv That's what I would expect as well, so it requires some scope in which you assume that no net new objects were added. That is no different from looking at the total ref count, where you also need such a scope to interpret an increased number as an issue.

I'll start adding a pytest integration to the yagot project. Consider this issue withdrawn.

[SVilgelm]

Does anyone know how to install python-debug on macOS?