search

eukreign / python-v8

Apache License 2.0
12 stars 6 forks source link

Refcount / Garbage collection segfaulting issues #10

Open eukreign opened 10 years ago

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 08, 2013 00:57:11

Continuation of: http://code.google.com/p/pyv8/source/detail?r=505 What steps will reproduce the problem? 1. raise an unhandled exception inside the testDestructor test What is the expected output? What do you see instead? A segfault What version of the product are you using? On what operating system? PyV8 revision 507 , Python3.3, Linux Please provide any additional information below. It's curious that the test has changed from asserting a refcount of 3 to 4. Inside the testDestructor you'll note that running JSEngine.collect() immediately after fn(obj) will lower the refcount to only 3 , not back to 2. I'm not sure what this means?

Here's the gdb backtrace.

(gdb) run build/lib.linux-x86_64-3.3/PyV8.py Starting program: /usr/bin/python3.3 build/lib.linux-x86_64-3.3/PyV8.py [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7ffff7ff1700 (LWP 9488)] ......................[New Thread 0x7ffff49ab700 (LWP 9489)] [New Thread 0x7fffeffff700 (LWP 9490)] [Thread 0x7fffeffff700 (LWP 9490) exited] [Thread 0x7ffff49ab700 (LWP 9489) exited] .[New Thread 0x7ffff49ab700 (LWP 9491)] .......4 F...................[Thread 0x7ffff49ab700 (LWP 9491) exited]

Program received signal SIGSEGV, Segmentation fault. 0x000000000042f976 in ?? () (gdb) bt

0 0x000000000042f976 in ?? ()

1 0x00007ffff51dc337 in ~object_base (this=, __in_chrg=)

at /home/nick/py3/boost_1_53_0/boost/python/object_core.hpp:526

2 ~object (this=0xb7c840, __in_chrg=) at /home/nick/py3/boost_1_53_0/boost/python/object_core.hpp:318

3 ~object (this=0xb7c840, __in_chrg=) at src/Wrapper.cpp:1732

4 ~auto_ptr (this=, __in_chrg=) at /usr/include/c++/4.6/backward/auto_ptr.h:170

5 ObjectTracer::~ObjectTracer (this=, __in_chrg=) at src/Wrapper.cpp:1732

6 0x00007ffff51dc401 in ~auto_ptr (this=, __in_chrg=) at /usr/include/c++/4.6/backward/auto_ptr.h:170

7 ContextTracer::~ContextTracer (this=0xc9ce60, __in_chrg=) at src/Wrapper.cpp:1833

8 0x00007ffff51dc491 in ContextTracer::WeakCallback (value=..., parameter=0xc9ce60) at src/Wrapper.cpp:1838

9 0x00007ffff52b8408 in v8::internal::GlobalHandles::PostGarbageCollectionProcessing(v8::internal::GarbageCollector, v8::internal::GCTracer*) () from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so

10 0x00007ffff52d5693 in v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GCTracer*) ()

from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so

11 0x00007ffff52d6f6e in v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollector, char const, char const) () from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so

12 0x00007ffff538dc3f in v8::internal::JSObject::UpdateMapCodeCache(v8::internal::Handlev8::internal::JSObject, v8::internal::Handlev8::internal::String, v8::internal::Handlev8::internal::Code) ()

from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so

13 0x00007ffff543087d in v8::internal::StubCache::ComputeLoadGlobal(v8::internal::Handlev8::internal::String, v8::internal::Handlev8::internal::JSObject, v8::internal::Handlev8::internal::GlobalObject, v8::internal::Handlev8::internal::JSGlobalPropertyCell, bool) ()

from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so

14 0x00007ffff530f70a in v8::internal::LoadIC::ComputeLoadMonomorphic(v8::internal::LookupResult*, v8::internal::Handlev8::internal::JSObject, v8::internal::Handlev8::internal::String) () from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so

15 0x00007ffff531280d in v8::internal::LoadIC::UpdateCaches(v8::internal::LookupResult*, v8::internal::InlineCacheState, v8::internal::Handlev8::internal::Object, v8::internal::Handlev8::internal::String) ()

from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so ---Type to continue, or q to quit---

16 0x00007ffff5313124 in v8::internal::LoadIC::Load(v8::internal::InlineCacheState, v8::internal::Handlev8::internal::Object, v8::internal::Handlev8::internal::String) () from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so

17 0x00007ffff5313381 in v8::internal::LoadIC_Miss(v8::internal::Arguments, v8::internal::Isolate*) ()

from /home/nick/py3/pyv8/build/lib.linux-x86_64-3.3/_PyV8.cpython-33m.so

18 0x00003852b78062ee in ?? ()

19 0x00003852b7806241 in ?? ()

20 0x00007fffffffb9f0 in ?? ()

21 0x00007fffffffba38 in ?? ()

22 0x00003852b78837e6 in ?? ()

23 0x00002e95e7209a71 in ?? ()

24 0x000005a8a53929f1 in ?? ()

25 0x000032b4656081d1 in ?? ()

26 0x000032b4656081d1 in ?? ()

27 0x000005a8a537b2d9 in ?? ()

28 0x00007fffffffba78 in ?? ()

29 0x00003852b7825b04 in ?? ()

30 0x000032b4656082a1 in ?? ()

31 0x000005a8a5392b11 in ?? ()

32 0x000032b4656081d1 in ?? ()

33 0x00003852b7825a41 in ?? ()

34 0x0000000800000000 in ?? ()

35 0x0000000000000000 in ?? ()

(gdb)

Original issue: http://code.google.com/p/pyv8/issues/detail?id=163

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 07, 2013 22:06:38

It seems that the JSEngine.collect() call is required to stop the segfault.

The exceptions were of course stopping this at the end of the call.

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 07, 2013 22:08:41

Compare

def testDestructor(self):
    JSEngine.collect()
    return

vs

def testDestructor(self):
    return
eukreign commented 10 years ago

From ndudfi...@gmail.com on February 07, 2013 22:48:25

In general the test suite seems to be sensitive to test execution ordering

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 07, 2013 22:51:39

http://code.google.com/p/pyv8/issues/detail?id=108

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 08, 2013 01:14:09

Interestingly, you can see that running the single failing test case works fine in isolation.

➜ pyv8 python3.3 build/lib.linux-x86_64-3.3/PyV8.py -v TestWrapper.testDestructor
2013-02-08 16:12:19,811 INFO testing PyV8 module 1.0 with V8 v3.16.13 testDestructor (main.TestWrapper) ... ok

Ran 1 test in 0.017s

OK

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 08, 2013 01:18:46

Try the following test suites, in precise order (pass as sys.argv)

TestContext TestDebug TestEngine TestWrapper TestDebug TestEngine TestMultithread TestWrapper

Those combinations/sequences cause segfaults ( along with many others, but these have the property of being short and sorted )

eukreign commented 10 years ago

From flier...@gmail.com on February 08, 2013 03:56:45

Status: Accepted
Labels: OpSys-All

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 08, 2013 04:17:58

@flier

Dunno, if this is of any interest to you:

def testDestructor(self):
    """

    This illustrates some weird behaviour with dangling references that
    haven't been explicitly `del` eted

    Seemingly other test methods aren't cleaning up and causing similar
    issues.

    """

    with JSContext() as reference_that_cause_this_test_to_fail:
        "The __exit__ deletes self but in this frame reference"
        "dangles thus Dispose aint called"

    class Hello(object):
        deleted = False
        def say(self): pass

        def __del__(self):
            Hello.deleted = True

    # We won't put this in a test() closure with automatic
    # deletion, cause we want to be explicit about what references
    # we are deleting
    with JSContext() as ctxt:
        fn = ctxt.eval("b = function (obj) { obj.say(); }; b")
        obj = Hello()

        self.assertEqual(2, sys.getrefcount(obj))
        fn(obj)
        self.assertEqual(4, sys.getrefcount(obj))

    self.assertFalse(Hello.deleted)

    # You MUST delete ALL these references
    del fn
    del obj
    del ctxt

    # Try and collect
    JSEngine.collect(force=True)

    # What's interesting is at this point it's STILL not deleted
    # (assuming reference_that_cause_this_test_to_fail is uncommented)
    self.assertFalse(Hello.deleted)

    # Now by deleting THIS reference, then running the collector
    # finally our __del__ method gets ran
    del reference_that_cause_this_test_to_fail
    JSEngine.collect(force=True)

    # Yes, it has definitely been ran
    self.assertTrue(Hello.deleted)
eukreign commented 10 years ago

From flier...@gmail.com on February 08, 2013 06:23:04

Thanks for your hints, it seems that two context have week relation, I will analyze it more depth later.

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 08, 2013 06:25:40

Do you have a donation link? :)

eukreign commented 10 years ago

From flier...@gmail.com on February 08, 2013 06:28:03

Thanks for your kindness, I'm working just for fun, or maybe later :)

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 08, 2013 06:44:06

Probably a dream, but would be nice if the many thousands of ZenCoding/Emmet users wanting PyV8 for sublime 3 would contribute to your project :)

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 09, 2013 01:52:24

@Flier

Just a follow up. We aren't currently using the SUPPORT_TRACE_LIFECYCLE stuff so undefined it and now we are seeing some stability!

// // Trace the object lifecycle // // #define SUPPORT_TRACE_LIFECYCLE 1

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 09, 2013 02:05:11

Is this trace life cycle code needed ?

If I'm understanding correctly it's mostly for development/troubleshooting diagnostics?

eukreign commented 10 years ago

From flier...@gmail.com on February 09, 2013 02:13:20

In fact, the object tracer will improve the performance and reduce the memory footprint, because the same JS object will use same Python wrapper object.

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 09, 2013 02:18:11

Thanks for the clarification. Yes, I just before saw the now failing test for object caching :)

Just undefining the trace macro seems to avoid the segfaults, which allows development to continue. Before, PyV8 was crashing the plugin_host process for ST3

I'm just a script kiddy with enough C knowledge to be dangerous, so I can only hope for youto remedy the sefaults in tracing code :)

eukreign commented 10 years ago

From flier...@gmail.com on February 09, 2013 03:29:59

Thanks for your patient, I will check it later, because today is the Chinese new year, so I must pay more attention to my family :P

eukreign commented 10 years ago

From ndudfi...@gmail.com on February 09, 2013 03:30:54

Enjoy your holiday :)

eukreign commented 10 years ago

From ndudfi...@gmail.com on June 15, 2013 23:55:04

Any update on this?