test_code fails under -j always due to an extra reference being held

[GoogleCodeExporter]

If you run Lib/test/test_code.py under -j always, the weakref tests fail. 
I've been trying to track this down over break, but it hasn't yielded so
I'm opening this bug in case anyone else can chime in with the answer.

To reproduce this more quickly, you can just put
_llvm.set_jit_control("always") at the top of the failing test case and it
will still fail (at least it does for me).

The clues I have so far:

The gc module says there are no referrers to the code object, so it must be
held by a non-PyObject, or we have a refleak.

The only code compiled during the test case is the code object constructed
by the exec statement.  Nothing else is compiled.

I think the first step is to figure out where we incref code objects when
compiling code objects, but I can't seem to find any such places.

Original issue reported on code.google.com by reid.kle...@gmail.com on 29 Dec 2009 at 3:53

[GoogleCodeExporter]

Aha, I think I found it.  Consider the following code:
def make_f():
    def f(): pass
    return f

This has the bytecode:
  2           0 LOAD_CONST               1 (<code object f at 0x38530, file
"<stdin>", line 2>)
              3 MAKE_FUNCTION            0
              6 STORE_FAST               0 (f)

  3           9 LOAD_FAST                0 (f)
             12 RETURN_VALUE        

The code object for the new f being created is a local constant of the code 
object
that makes f.  I looked at LOAD_CONST in the function builder, and it uses
ConstantMirror::GetGlobalVariableFor(PyObject *), which uses the complex value 
handle
mechanism that Jeff created to incref globals to preserve them until after the 
LLVM
global variable has become unused.  If I forcibly collect unused globals from 
GDB
while stopped after the exec, the test passes.

In order to make the test pass under -j always, I think we need to expose a way 
to
collect globals to Python code somehow.  We could do it by adding a method to 
the
_llvm module, but that's gross.  I was thinking that if someone explicitly calls
gc.collect, then it should also collect dead globals.  Does that sound 
appropriate? 
I'll send out a change soon that does that.

Original comment by reid.kle...@gmail.com on 29 Dec 2009 at 4:18

[GoogleCodeExporter]

This was fixed in r964, right?

Original comment by collinw on 11 Jan 2010 at 7:43

[GoogleCodeExporter]

Yup.

Original comment by reid.kle...@gmail.com on 11 Jan 2010 at 10:41

• Changed state: Fixed