Refcount / Garbage collection segfaulting issues

[GoogleCodeExporter]

Continuation of: http://code.google.com/p/pyv8/source/detail?r=505

What steps will reproduce the problem?
1. raise an unhandled exception inside the testDestructor test

What is the expected output? What do you see instead?
A segfault

What version of the product are you using? On what operating system?
PyV8 revision 507, Python3.3, Linux

Please provide any additional information below.

It's curious that the test has changed from asserting a refcount of 3 to 4. 
Inside the `testDestructor` you'll note that running `JSEngine.collect()` 
immediately after `fn(obj)` will lower the refcount to only 3 , not back to 2. 
I'm not sure what this means?

Here's the gdb backtrace.

(gdb) run build/lib.linux-x86_64-3.3/PyV8.py 
Starting program: /usr/bin/python3.3 build/lib.linux-x86_64-3.3/PyV8.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7ff1700 (LWP 9488)]
......................[New Thread 0x7ffff49ab700 (LWP 9489)]
[New Thread 0x7fffeffff700 (LWP 9490)]
[Thread 0x7fffeffff700 (LWP 9490) exited]
[Thread 0x7ffff49ab700 (LWP 9489) exited]
.[New Thread 0x7ffff49ab700 (LWP 9491)]
.......4
F...................[Thread 0x7ffff49ab700 (LWP 9491) exited]

Program received signal SIGSEGV, Segmentation fault.
0x000000000042f976 in ?? ()
(gdb) bt
#0  0x000000000042f976 in ?? ()
#1  0x00007ffff51dc337 in ~object_base (this=<optimized out>, 
__in_chrg=<optimized out>)
    at /home/nick/py3/boost_1_53_0/boost/python/object_core.hpp:526
#2  ~object (this=0xb7c840, __in_chrg=<optimized out>) at 
/home/nick/py3/boost_1_53_0/boost/python/object_core.hpp:318
#3  ~object (this=0xb7c840, __in_chrg=<optimized out>) at src/Wrapper.cpp:1732
#4  ~auto_ptr (this=<optimized out>, __in_chrg=<optimized out>) at 
/usr/include/c++/4.6/backward/auto_ptr.h:170
#5  ObjectTracer::~ObjectTracer (this=<optimized out>, __in_chrg=<optimized 
out>) at src/Wrapper.cpp:1732
#6  0x00007ffff51dc401 in ~auto_ptr (this=<synthetic pointer>, 
__in_chrg=<optimized out>) at /usr/include/c++/4.6/backward/auto_ptr.h:170
#7  ContextTracer::~ContextTracer (this=0xc9ce60, __in_chrg=<optimized out>) at 
src/Wrapper.cpp:1833
#8  0x00007ffff51dc491 in ContextTracer::WeakCallback (value=..., 
parameter=0xc9ce60) at src/Wrapper.cpp:1838
#9  0x00007ffff52b8408 in 
v8::internal::GlobalHandles::PostGarbageCollectionProcessing(v8::internal::Garba
geCollector, v8::internal::GCTracer*) () from 
/home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
#10 0x00007ffff52d5693 in 
v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, 
v8::internal::GCTracer*) ()
   from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
#11 0x00007ffff52d6f6e in 
v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, 
v8::internal::GarbageCollector, char const*, char const*) () from 
/home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
#12 0x00007ffff538dc3f in 
v8::internal::JSObject::UpdateMapCodeCache(v8::internal::Handle<v8::internal::JS
Object>, v8::internal::Handle<v8::internal::String>, 
v8::internal::Handle<v8::internal::Code>) ()
   from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
#13 0x00007ffff543087d in 
v8::internal::StubCache::ComputeLoadGlobal(v8::internal::Handle<v8::internal::St
ring>, v8::internal::Handle<v8::internal::JSObject>, 
v8::internal::Handle<v8::internal::GlobalObject>, 
v8::internal::Handle<v8::internal::JSGlobalPropertyCell>, bool) ()
   from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
#14 0x00007ffff530f70a in 
v8::internal::LoadIC::ComputeLoadMonomorphic(v8::internal::LookupResult*, 
v8::internal::Handle<v8::internal::JSObject>, 
v8::internal::Handle<v8::internal::String>) () from 
/home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
#15 0x00007ffff531280d in 
v8::internal::LoadIC::UpdateCaches(v8::internal::LookupResult*, 
v8::internal::InlineCacheState, v8::internal::Handle<v8::internal::Object>, 
v8::internal::Handle<v8::internal::String>) ()
   from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
---Type <return> to continue, or q <return> to quit---
#16 0x00007ffff5313124 in 
v8::internal::LoadIC::Load(v8::internal::InlineCacheState, 
v8::internal::Handle<v8::internal::Object>, 
v8::internal::Handle<v8::internal::String>) () from 
/home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
#17 0x00007ffff5313381 in v8::internal::LoadIC_Miss(v8::internal::Arguments, 
v8::internal::Isolate*) ()
   from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so
#18 0x00003852b78062ee in ?? ()
#19 0x00003852b7806241 in ?? ()
#20 0x00007fffffffb9f0 in ?? ()
#21 0x00007fffffffba38 in ?? ()
#22 0x00003852b78837e6 in ?? ()
#23 0x00002e95e7209a71 in ?? ()
#24 0x000005a8a53929f1 in ?? ()
#25 0x000032b4656081d1 in ?? ()
#26 0x000032b4656081d1 in ?? ()
#27 0x000005a8a537b2d9 in ?? ()
#28 0x00007fffffffba78 in ?? ()
#29 0x00003852b7825b04 in ?? ()
#30 0x000032b4656082a1 in ?? ()
#31 0x000005a8a5392b11 in ?? ()
#32 0x000032b4656081d1 in ?? ()
#33 0x00003852b7825a41 in ?? ()
#34 0x0000000800000000 in ?? ()
#35 0x0000000000000000 in ?? ()
(gdb) 

Original issue reported on code.google.com by ndudfi...@gmail.com on 8 Feb 2013 at 5:57

[GoogleCodeExporter]

It seems that the JSEngine.collect() call is required to stop the segfault.

The exceptions were of course stopping this at the end of the call.

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 6:06

[GoogleCodeExporter]

Compare

    def testDestructor(self):
        JSEngine.collect()
        return

vs

    def testDestructor(self):
        return

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 6:08

[GoogleCodeExporter]

In general the test suite seems to be sensitive to test execution ordering

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 6:48

[GoogleCodeExporter]

http://code.google.com/p/pyv8/issues/detail?id=108

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 6:51

[GoogleCodeExporter]

Interestingly, you can see that running the single failing test case works fine 
in isolation.

➜  pyv8  python3.3 build/lib.linux-x86_64-3.3/PyV8.py -v   
TestWrapper.testDestructor          
2013-02-08 16:12:19,811 INFO testing PyV8 module 1.0 with V8 v3.16.13
testDestructor (__main__.TestWrapper) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.017s

OK

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 9:14

[GoogleCodeExporter]

Try the following test suites, in precise order (pass as sys.argv)

TestContext TestDebug TestEngine TestWrapper
TestDebug TestEngine TestMultithread TestWrapper

Those combinations/sequences cause segfaults ( along with many others, but 
these have the property of being short and `sorted` )

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 9:18

[GoogleCodeExporter]

Original comment by flier...@gmail.com on 8 Feb 2013 at 11:56

• Changed state: Accepted
• Added labels: OpSys-All

[GoogleCodeExporter]

@flier

Dunno, if this is of any interest to you:

    def testDestructor(self):
        """

        This illustrates some weird behaviour with dangling references that
        haven't been explicitly `del` eted

        Seemingly other test methods aren't cleaning up and causing similar
        issues.

        """

        with JSContext() as reference_that_cause_this_test_to_fail:
            "The __exit__ deletes self but in this frame reference"
            "dangles thus Dispose aint called"

        class Hello(object):
            deleted = False
            def say(self): pass

            def __del__(self):
                Hello.deleted = True

        # We won't put this in a test() closure with automatic
        # deletion, cause we want to be explicit about what references
        # we are deleting
        with JSContext() as ctxt:
            fn = ctxt.eval("b = function (obj) { obj.say(); }; b")
            obj = Hello()

            self.assertEqual(2, sys.getrefcount(obj))
            fn(obj)
            self.assertEqual(4, sys.getrefcount(obj))

        self.assertFalse(Hello.deleted)

        # You MUST delete ALL these references
        del fn
        del obj
        del ctxt

        # Try and collect
        JSEngine.collect(force=True)

        # What's interesting is at this point it's STILL not deleted
        # (assuming reference_that_cause_this_test_to_fail is uncommented)
        self.assertFalse(Hello.deleted)

        # Now by deleting THIS reference, then running the collector
        # finally our __del__ method gets ran
        del reference_that_cause_this_test_to_fail
        JSEngine.collect(force=True)

        # Yes, it has definitely been ran
        self.assertTrue(Hello.deleted)

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 12:17

[GoogleCodeExporter]

Thanks for your hints, it seems that two context have week relation, I will 
analyze it more depth later.

Original comment by flier...@gmail.com on 8 Feb 2013 at 2:23

[GoogleCodeExporter]

Do you have a donation link? :)

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 2:25

[GoogleCodeExporter]

Thanks for your kindness, I'm working just for fun, or maybe later :)

Original comment by flier...@gmail.com on 8 Feb 2013 at 2:28

[GoogleCodeExporter]

Probably a dream, but would be nice if the many thousands of ZenCoding/Emmet 
users wanting PyV8 for sublime 3 would contribute to your project :)

Original comment by ndudfi...@gmail.com on 8 Feb 2013 at 2:44

[GoogleCodeExporter]

@Flier

Just a follow up. We aren't currently using the SUPPORT_TRACE_LIFECYCLE stuff 
so undefined it and now we are seeing some stability!

//
// Trace the object lifecycle
//
// #define SUPPORT_TRACE_LIFECYCLE 1

Original comment by ndudfi...@gmail.com on 9 Feb 2013 at 9:52

[GoogleCodeExporter]

Is this trace life cycle code needed ?  

If I'm understanding correctly it's mostly for development/troubleshooting 
diagnostics?

Original comment by ndudfi...@gmail.com on 9 Feb 2013 at 10:05

[GoogleCodeExporter]

In fact, the object tracer will improve the performance and reduce the memory 
footprint, because the same JS object will use same Python wrapper object.

Original comment by flier...@gmail.com on 9 Feb 2013 at 10:13

[GoogleCodeExporter]

Thanks for the clarification. Yes, I just before saw the now failing test for 
object caching :)

Just undefining the trace macro seems to avoid the segfaults, which allows 
development to continue. Before, PyV8 was crashing the plugin_host process for 
ST3

I'm just a script kiddy with enough C knowledge to be dangerous, so I can only 
hope for youto remedy the sefaults in tracing code :)

Original comment by ndudfi...@gmail.com on 9 Feb 2013 at 10:18

[GoogleCodeExporter]

Thanks for your patient, I will check it later, because today is the Chinese 
new year, so I must pay more attention to my family :P

Original comment by flier...@gmail.com on 9 Feb 2013 at 11:29

[GoogleCodeExporter]

Enjoy your holiday :)

Original comment by ndudfi...@gmail.com on 9 Feb 2013 at 11:30

[GoogleCodeExporter]

Any update on this?

Original comment by ndudfi...@gmail.com on 16 Jun 2013 at 6:55