search

jeanphix / Ghost.py

Webkit based scriptable web browser for python.
http://ghost-py.readthedocs.org/en/latest/
2.77k stars 380 forks source link

segmentation faults #186

Open hitigon opened 9 years ago

hitigon commented 9 years ago

The program will exit with segmentation faults when loading some pages. But sometimes, these pages may be loaded normally and the program exits normally.

Report:

Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: EXC_I386_GPFLT

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   QtNetwork                       0x000000010854aecf QNetworkAccessHttpBackend::loadFromCacheIfAllowed(QHttpNetworkRequest&) + 415
1   QtNetwork                       0x000000010854d470 QNetworkAccessHttpBackend::postRequest() + 7120
2   QtNetwork                       0x0000000108542547 QNetworkAccessBackend::start() + 823
3   QtNetwork                       0x000000010855dca4 QNetworkReplyImplPrivate::_q_startOperation() + 84
4   QtNetwork                       0x000000010856073d QNetworkReplyImplPrivate::setup(QNetworkAccessManager::Operation, QNetworkRequest const&, QIODevice*) + 1037
5   QtNetwork                       0x000000010853e6de QNetworkAccessManager::createRequest(QNetworkAccessManager::Operation, QNetworkRequest const&, QIODevice*) + 1022
6   QtNetwork                       0x000000010853b70c QNetworkAccessManager::get(QNetworkRequest const&) + 28
7   QtWebKit                        0x000000010932c6aa WebCore::QNetworkReplyHandler::sendNetworkRequest(QNetworkAccessManager*, WebCore::ResourceRequest const&) + 970
8   QtWebKit                        0x000000010932c955 WebCore::QNetworkReplyHandler::start() + 101
9   QtWebKit                        0x0000000109328869 WebCore::QNetworkReplyHandlerCallQueue::flush() + 169
10  QtWebKit                        0x0000000109328939 WebCore::QNetworkReplyHandlerCallQueue::push(void (WebCore::QNetworkReplyHandler::*)()) + 41
11  QtWebKit                        0x0000000109328a63 WebCore::QNetworkReplyHandler::QNetworkReplyHandler(WebCore::ResourceHandle*, WebCore::QNetworkReplyHandler::LoadType, bool) + 291
12  QtWebKit                        0x0000000109325c34 WebCore::ResourceHandle::start(WebCore::NetworkingContext*) + 164
13  QtWebKit                        0x00000001091b3a1c WebCore::ResourceHandle::create(WebCore::NetworkingContext*, WebCore::ResourceRequest const&, WebCore::ResourceHandleClient*, bool, bool) + 220
14  QtWebKit                        0x00000001090b4b11 WebCore::PingLoader::PingLoader(WebCore::Frame*, WebCore::ResourceRequest&) + 209
15  QtWebKit                        0x00000001090b5624 WebCore::PingLoader::loadImage(WebCore::Frame*, WebCore::KURL const&) + 612
16  QtWebKit                        0x000000010907bee6 WebCore::CachedResourceLoader::requestImage(WTF::String const&) + 438
17  QtWebKit                        0x00000001090ae9e4 WebCore::ImageLoader::updateFromElement() + 852
18  QtWebKit                        0x0000000108f71a89 WebCore::HTMLImageElement::parseMappedAttribute(WebCore::Attribute*) + 505
19  QtWebKit                        0x0000000108e1f936 WebCore::StyledElement::attributeChanged(WebCore::Attribute*, bool) + 150
20  QtWebKit                        0x0000000108dec267 WebCore::NamedNodeMap::addAttribute(WTF::PassRefPtr<WebCore::Attribute>) + 119
21  QtWebKit                        0x0000000108dd606d WebCore::Element::setAttribute(WebCore::QualifiedName const&, WTF::AtomicString const&, int&) + 861
22  QtWebKit                        0x0000000108dd6411 WebCore::Element::setAttribute(WebCore::QualifiedName const&, WTF::AtomicString const&) + 17
23  QtWebKit                        0x00000001088fc7ea WebCore::setJSHTMLImageElementSrc(JSC::ExecState*, JSC::JSObject*, JSC::JSValue) + 90
24  QtWebKit                        0x00000001088fb79a WebCore::JSHTMLImageElement::put(JSC::ExecState*, JSC::Identifier const&, JSC::JSValue, JSC::PutPropertySlot&) + 186
25  QtWebKit                        0x00000001096781bc cti_op_put_by_id_generic + 108
26  ???                             0x00003de68803f7ed 0 + 68060333733869
27  QtWebKit                        0x00000001096398fd JSC::Interpreter::executeCall(JSC::ExecState*, JSC::JSObject*, JSC::CallType, JSC::CallData const&, JSC::JSValue, JSC::ArgList const&) + 1117
28  QtWebKit                        0x00000001096d7a91 JSC::call(JSC::ExecState*, JSC::JSValue, JSC::CallType, JSC::CallData const&, JSC::JSValue, JSC::ArgList const&) + 49
29  QtWebKit                        0x0000000108c6bbb3 WebCore::JSEventListener::handleEvent(WebCore::ScriptExecutionContext*, WebCore::Event*) + 2179
30  QtWebKit                        0x0000000108ddfad0 WebCore::EventTarget::fireEventListeners(WebCore::Event*, WebCore::EventTargetData*, WTF::Vector<WebCore::RegisteredEventListener, 1ul>&) + 240
31  QtWebKit                        0x0000000108ddfea2 WebCore::EventTarget::fireEventListeners(WebCore::Event*) + 146
32  QtWebKit                        0x00000001091015f6 WebCore::DOMWindow::dispatchEvent(WTF::PassRefPtr<WebCore::Event>, WTF::PassRefPtr<WebCore::EventTarget>) + 278
33  QtWebKit                        0x0000000109098cc6 WebCore::FrameLoader::stopLoading(WebCore::UnloadEventPolicy) + 438
34  QtWebKit                        0x000000010909945d WebCore::FrameLoader::closeURL() + 61
35  QtWebKit                        0x00000001090994c9 WebCore::FrameLoader::detachFromParent() + 57
36  QtWebKit                        0x0000000108bb9732 QWebPage::~QWebPage() + 66
37  QtWebKit.so                     0x00000001087048f7 QWebPageWrapper::~QWebPageWrapper() + 55
38  QtCore                          0x00000001055208ef QObjectPrivate::deleteChildren() + 95
39  QtCore                          0x0000000105529d57 QObject::~QObject() + 1655
40  QtGui                           0x0000000107558b05 QApplication::~QApplication() + 1413
41  QtGui.so                        0x000000010691a857 QApplicationWrapper::~QApplicationWrapper() + 55
42  libpyside-python2.7.1.2.dylib   0x0000000105339a90 PySide::destroyQCoreApplication() + 128
43  libpyside-python2.7.1.2.dylib   0x00000001053399f2 PySide::runCleanupFunctions() + 130
44  QtCore.so                       0x0000000105158943 SbkQtCoreModule___moduleShutdown(_object*) + 19
45  org.python.python               0x0000000104ad212f PyEval_EvalFrameEx + 16242
46  org.python.python               0x0000000104ace093 PyEval_EvalCodeEx + 1641
47  org.python.python               0x0000000104a75796 0x104a4e000 + 161686
48  org.python.python               0x0000000104a57f72 PyObject_Call + 101
49  org.python.python               0x0000000104ad401f PyEval_CallObjectWithKeywords + 93
50  org.python.python               0x0000000104aec068 Py_Finalize + 184
51  org.python.python               0x0000000104afd40d Py_Main + 3165
52  libdyld.dylib                   0x00007fff909565fd start + 1
hitigon commented 9 years ago

I found that qt events are still running after the main frame has been loaded, when the Python is ready for exiting, processEvents is sending requests to somewhere. This causes the segment fault on exit. Not familiar with QT, any suggests?

update: I may not be quite right about processEvents, but there is something still running in the Qt application. I use os._exit(0) to force the program to quit to fix this problem for now.

kravietz commented 9 years ago

I see the same stuff quite frequently. Unfortunately it seems to be in WebKit itself, not in any of the Python libs. Also I can't find any constant pattern - some sites crash on the first run, then they pass OK on the second run.

jeanphix commented 9 years ago

@kravietz Could you paste few links that chrash? Are you also running osX?

hitigon commented 9 years ago

@kravietz You are right, after running my program for almost one month, the crashes were most likely random.

kravietz commented 9 years ago

Nope, it's crashing on Linux. Sample:

DEBUG 2014-11-29 14:02:18,580 tasks Worker luxemburg fetching URL http://www.aleqt.com/2014/11/28/article_910177.html date fetched 2014-11-29 14:02:18.566997+00:00
Fatal Python error: Segmentation fault

Current thread 0x00007fe8a316a740 (most recent call first):
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/Ghost.py-0.1b3-py2.7.egg/ghost/ghost.py", line 911 in wait_for
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/Ghost.py-0.1b3-py2.7.egg/ghost/ghost.py", line 928 in wait_for_page_loaded
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/Ghost.py-0.1b3-py2.7.egg/ghost/ghost.py", line 689 in open
  File "/home/webcookies/wps2/register_site/tasks.py", line 252 in FetchWebsite
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/celery/app/trace.py", line 438 in __protected_call__
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/celery/app/trace.py", line 240 in trace_task
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/celery/app/trace.py", line 349 in _fast_trace_task
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/billiard/pool.py", line 361 in workloop
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/billiard/pool.py", line 289 in run
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/billiard/process.py", line 292 in _bootstrap
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/billiard/forking.py", line 464 in main
  File "<string>", line 1 in <module>
^[[1;31m[2014-11-29 14:02:25,542: ERROR/MainProcess] Process 'Worker-374' pid:32542 exited with 'signal 11 (SIGSEGV)'^[[0m
^[[1;31m[2014-11-29 14:02:25,563: ERROR/MainProcess] Task register_site.tasks.FetchWebsite[cff1ee3d-89ed-480d-b9c6-e6717c91a1a2] raised unexpected: WorkerLostError('Worker exited prematurely: signal 11 (SIGSEGV).',)
Traceback (most recent call last):
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/billiard/pool.py", line 1169, in mark_as_worker_lost
    human_status(exitcode)),
WorkerLostError: Worker exited prematurely: signal 11 (SIGSEGV).

And then testing the same in command line:

$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ghost import Ghost
>>> g=Ghost()
>>> g.open('http://www.aleqt.com/2014/11/28/article_910177.html')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/Ghost.py-0.1b4-py2.7.egg/ghost/ghost.py", line 689, in open
    return self.wait_for_page_loaded()
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/Ghost.py-0.1b4-py2.7.egg/ghost/ghost.py", line 928, in wait_for_page_loaded
    'Unable to load requested page')
  File "/home/webcookies/wps2/local/lib/python2.7/site-packages/Ghost.py-0.1b4-py2.7.egg/ghost/ghost.py", line 909, in wait_for
    raise TimeoutError(timeout_message)
ghost.ghost.TimeoutError: Unable to load requested page
>>> g.open('http://www.aleqt.com/2014/11/28/article_910177.html')
Segmentation fault (core dumped)
(wps2)kravietz@luxemburg:/home/webcookies/wps2$ curl -s http://www.aleqt.com/2014/11/28/article_910177.html | wc -l
1767

Kernel says:

[20061.210061] traps: python[23202] general protection ip:7fa2a1e09787 sp:7fffd9c73db0 error:0 in libQtWebKit.so.4.10.2[7fa2a092a000+1fc9000]

Actually, this particular page seems to be a good target for testing - out of 3 attempts one time it loaded OK, and twice it crashed with segmentationi fault...

kravietz commented 9 years ago

Actually, I think I've found it:

#0  0x00007fcf592fa787 in QtMIMETypeSniffer::~QtMIMETypeSniffer() () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#1  0x00007fcf582d0a0c in WebCore::JSNodeOwner::isReachableFromOpaqueRoots(JSC::Handle<JSC::Unknown>, void*, JSC::SlotVisitor&) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#2  0x00007fcf593fbfea in JSC::WeakBlock::visit(JSC::HeapRootVisitor&) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#3  0x00007fcf5940d4ab in JSC::MarkedSpace::visitWeakSets(JSC::HeapRootVisitor&) ()
   from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#4  0x00007fcf593ff086 in JSC::Heap::markRoots(bool) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#5  0x00007fcf59403e85 in JSC::Heap::collect(JSC::Heap::SweepToggle) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#6  0x00007fcf595f2492 in JSC::DefaultGCActivityCallback::doWork() () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#7  0x00007fcf594071b9 in JSC::HeapTimer::timerEvent(QTimerEvent*) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#8  0x00007fcf5fe1ca31 in QObject::event(QEvent*) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#9  0x00007fcf5d6c4e2c in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
   from /usr/lib/x86_64-linux-gnu/libQtGui.so.4
#10 0x00007fcf5d6cb4a0 in QApplication::notify(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQtGui.so.4
#11 0x00007fcf5e396c5e in ?? () from /usr/lib/python2.7/dist-packages/PySide/QtGui.so
#12 0x00007fcf5fe044dd in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#13 0x00007fcf5fe34323 in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#14 0x00007fcf5fe315f1 in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#15 0x00007fcf5f4a4e04 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#16 0x00007fcf5f4a5048 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#17 0x00007fcf5f4a50ec in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#18 0x00007fcf5fe317a1 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
   from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#19 0x00007fcf5d766bb6 in ?? () from /usr/lib/x86_64-linux-gnu/libQtGui.so.4
#20 0x00007fcf5fe0801f in QCoreApplication::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
   from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#21 0x00007fcf60bc72c2 in ?? () from /usr/lib/python2.7/dist-packages/PySide/QtCore.so
#22 0x000000000052c6d5 in PyEval_EvalFrameEx ()
#23 0x000000000052cf32 in PyEval_EvalFrameEx ()
#24 0x000000000055c594 in PyEval_EvalCodeEx ()
#25 0x000000000052ca8d in PyEval_EvalFrameEx ()
#26 0x000000000055c594 in PyEval_EvalCodeEx ()
#27 0x000000000052ca8d in PyEval_EvalFrameEx ()
#28 0x000000000055c594 in PyEval_EvalCodeEx ()
#29 0x00000000005b7392 in PyEval_EvalCode ()

Will file a bug with WebKit for that.

jeanphix commented 9 years ago

@kravietz @hitigon Looks like you're running "outdated" ghosts, you should upgrade as https://github.com/jeanphix/Ghost.py/commit/3044c63e10a3ee3b239c68c93b80a55143784259 from 0.1b5 fixes lot of issues related to cache.

HarryQian commented 9 years ago

I am also troubled with the core. For some pages, it always causes core dumping during page loading.

0 0x00007f70dc327787 in QtMIMETypeSniffer::~QtMIMETypeSniffer() () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4

1 0x00007f70db2fda0c in WebCore::JSNodeOwner::isReachableFromOpaqueRoots(JSC::HandleJSC::Unknown, void_, JSC::SlotVisitor&) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4

2 0x00007f70dc428fea in JSC::WeakBlock::visit(JSC::HeapRootVisitor&) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4

3 0x00007f70dc43a4ab in JSC::MarkedSpace::visitWeakSets(JSC::HeapRootVisitor&) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4

4 0x00007f70dc42c086 in JSC::Heap::markRoots(bool) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4

5 0x00007f70dc430e85 in JSC::Heap::collect(JSC::Heap::SweepToggle) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4

6 0x00007f70dc61f492 in JSC::DefaultGCActivityCallback::doWork() () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4

7 0x00007f70dc4341b9 in JSC::HeapTimer::timerEvent(QTimerEvent_) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4

8 0x00007f70e2362a31 in QObject::event(QEvent_) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

9 0x00007f70dfe90e2c in QApplicationPrivate::notifyhelper(QObject, QEvent_) () from /usr/lib/x86_64-linux-gnu/libQtGui.so.4

10 0x00007f70dfe974a0 in QApplication::notify(QObject, QEvent) () from /usr/lib/x86_64-linux-gnu/libQtGui.so.4

11 0x00007f70e0ebb186 in ?? () from /usr/lib/python2.7/dist-packages/PyQt4/QtGui.so

12 0x00007f70e234a4dd in QCoreApplication::notifyInternal(QObject_, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

13 0x00007f70e237a323 in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

14 0x00007f70e2377629 in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

15 0x00007f70e19f3e04 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

16 0x00007f70e19f4048 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

17 0x00007f70e19f40ec in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

18 0x00007f70e23777a1 in QEventDispatcherGlib::processEvents(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

19 0x00007f70dff32bb6 in ?? () from /usr/lib/x86_64-linux-gnu/libQtGui.so.4

20 0x00007f70e23490af in QEventLoop::processEvents(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

21 0x00007f70e23493a5 in QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

22 0x00007f70e234eb79 in QCoreApplication::exec() () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

23 0x00007f70e0e771f0 in ?? () from /usr/lib/python2.7/dist-packages/PyQt4/QtGui.so

24 0x000000000052c6d5 in PyEval_EvalFrameEx ()

25 0x000000000052cf32 in PyEval_EvalFrameEx ()

26 0x000000000055c594 in PyEval_EvalCodeEx ()

27 0x00000000005b7392 in PyEval_EvalCode ()

28 0x0000000000469663 in ?? ()

29 0x00000000004699e3 in PyRun_FileExFlags ()

30 0x0000000000469f1c in PyRun_SimpleFileExFlags ()

31 0x000000000046ab81 in Py_Main ()

32 0x00007f70e4568ec5 in __libc_start_main (main=0x46ac3f
, argc=9, argv=0x7fffac576b08, init=, fini=, rtld_fini=, stack_end=0x7fffac576af8)

at libc-start.c:287

33 0x000000000057497e in _start ()

(gdb) info threads Id Target Id Frame 7 Thread 0x7f708bfff700 (LWP 30545) 0x00007f70e4634c6d in poll () at ../sysdeps/unix/syscall-template.S:81 6 Thread 0x7f708ade2700 (LWP 30548) 0x00007f70e46398e3 in select () at ../sysdeps/unix/syscall-template.S:81 5 Thread 0x7f70916d0700 (LWP 30544) 0x00007f70e4634c6d in poll () at ../sysdeps/unix/syscall-template.S:81 4 Thread 0x7f708b7fe700 (LWP 30546) pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 3 Thread 0x7f7092426700 (LWP 30543) pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 2 Thread 0x7f70d4286700 (LWP 30541) pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

HarryQian commented 9 years ago

OK, I think I found the core-dumping case: When the page uses following HTML elements, it cause segment faults

  1. localStorage or sessionStorage
  2. websocket

the direct way to avoid them is to disable them

byaka commented 9 years ago

@HarryQian commented on 22 дек. 2014 г., 7:52 GMT+3:

OK, I think I found the core-dumping case:

When the page uses following HTML elements, it cause segment faults

  1. localStorage or sessionStorage
  2. websocket

the direct way to avoid them is to disable them

But how do this? A can't find anything about disabling features in documentation.

HarryQian commented 9 years ago

Now I am using PY-QT 5, it’s more efficient and really gets seldom core dump. so better update to new version.

byaka commented 9 years ago

I have Debian Wheezy on my servers... Installing Qt5 also update big part of system... Not good way.

petermat commented 9 years ago

Harry: ghost doesnt work with qt5, just qt4.

kravietz commented 9 years ago

@petermat Actually it does, with this small patch https://github.com/jeanphix/Ghost.py/pull/205 and you will also need Python3 as there's no PyQt5 for Python2 (at least not in Ubuntu). But it works suprisingly well:

Python 3.4.0 (default, Jun 19 2015, 14:20:21) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ghost import Ghost
>>> g=Ghost()
>>> g.open('https://ipsec.pl/')
(<ghost.ghost.HttpResource object at 0x7fdb723f2400>, ...

Unfortunately, this fatal test site http://www.aleqt.com/2014/11/28/article_910177.html still causes a crash inside Qt5 libraries :(

sqd commented 9 years ago

It keeps crashing randomly even on v0.2 ... Notably, the log

[WARNING ] QT: QFont::setPixelSize: Pixel size <= 0 (0)

seems to always precede the crash

Brainback commented 8 years ago

settting session.page = None fixes it for me

from ghost import Ghost

gho = Ghost()

with gho.start() as session:
     page, extra_resources = session.open("https://ipsec.pl/")
     assert page.http_status == 200
     session.page = None
byaka commented 8 years ago

for me help

   webkit=Ghost()
   webkit.evaluate('window.WebSockets=undefined')
   webkit.evaluate('window.localStorage=undefined')
   webkit.evaluate('window.sessionStorage=undefined')
   webkit.evaluate('window.RTCPeerConnection=undefined')
   webkit.evaluate('window.webkitRTCPeerConnection=undefined')
   webkit.evaluate('window.mozRTCPeerConnection=undefined')
knightliao commented 8 years ago

@Brainback if page is set to None, it will lead to a memory leak in the program. the memory will exhaust.

CJSoldier commented 8 years ago

I got the same problem. I pulled the docker image named ghost-py. The image have already installed python2.7 and pyside. I added ghost0.2.3. The docker runs on 64 bit ubuntu14.04. The same code ran on win7 with pyqt4 installed works well,but when ran in docker,it exited silently sometimes and I didn't catch any exception. Finally I found that the exception was "segmentation fault" or "qt - WARNING - ghost: Fatal IO error: client killed". If you solved the problem,please let me know.

d33tah commented 7 years ago

The real reason for having this bug is that you're using nonpatched qtwebkit. Try out the one from 2.3.2-0ubuntu9 (Ubuntu 16.04 or newer) and it should go away.

jeanphix commented 7 years ago

@d33tah please consider giving a try at this branch (PySide2, Qt5): https://github.com/jeanphix/ghost.py/tree/pyside2

Cheers,

d33tah commented 7 years ago

@jeanphix TBH this bug is not affecting me, it's just number one in Google search for this segfault. I wanted to let others know because I had the same problem in another application using qtwebkit.

jeanphix commented 7 years ago

@d33tah Sure, just I'd be happy to get feedback about the upcoming stuff :)

rcoh commented 7 years ago

@jeanphix I'm running pyside2 and still getting segfaults FYI

jeanphix commented 7 years ago

@rcoh When does it happen?

rcoh commented 7 years ago

It's happening on the session.open call. Here's my flow:

  1. Create ghost()
  2. Create new session. Open session using with, wait for a couple of things.
  3. Repeat 2
  4. Segfault on session.open

Possibly the problem is that I'm opening multiple sessions? Having multiple sessions running at the same time is critical for my use case, however.

jeanphix commented 7 years ago

@rcoh Can't reproduce it, can you write a tiny failing script?

rcoh commented 7 years ago

Hmm it appears to be related to the specific page I was loading...will have to debug more to figure out what about the page is causing the segfault. Is there anyway to get more of a stack trace?

rcoh commented 7 years ago

I think I found the problem. 

INFO:ghost.session:Page loaded https://[redacted]
INFO:ghost.session:Closing session
INFO:ghost.session:Resource loaded: https://[redacted] 200
INFO:loadtool:Start action: <_MainThread(MainThread, started 139801505380096)>
INFO:ghost.application:Initializing QT application
INFO:ghost.session:Starting new session
Segmentation fault

It seems like a resource is getting loaded even after the session is closed. When a attempted to open a new session, we get a segfault.

rcoh commented 7 years ago

This is fixed if I use a session.sleep() but it's pretty annoying

INFO:ghost.session:Page loaded https://....
(sleep for 20 seconds)
INFO:ghost.session:Closing session
INFO:ghost.session:Resource loaded: https://...
rcoh commented 7 years ago

Although since as a function of this, I can't run concurrent sessions anyway without a segfault, this doesn't really work for me.

jeanphix commented 7 years ago

@rcoh think it's related to this https://github.com/jeanphix/Ghost.py/pull/319/commits/52f31087c7d08fa7349ec9c55d75ff901be4f446, right ?

ping @EvaSDK

rcoh commented 7 years ago

yeah that looks like precisely the issue. Any idea about concurrent sessions?

jeanphix commented 7 years ago

@rcoh having several sessions living at same time just works well, isn't it?

rcoh commented 7 years ago

hmm I got immediate segfaults. Will try to make a reproducer

rcoh commented 7 years ago

@jeanphix https://gist.github.com/rcoh/d8c119441f0b2789a36cc3368e1273c0 I'm running that script against the pyside2 branch within the provided docker container.

EvaSDK commented 6 years ago

Completely missed this ping, sorry for this. I merged my network request tracker in ghost-py-0-2 branch this week as I was finally re-assigned to working on this. I am still tracking down some segfaults but it proves difficult.