Crashes on Linux

[cls-nebadje]

Hi Juan,

I'm currently on my Ubuntu 14.04 64bit workstation at work and running the linux build. Unfortunately it crashes pretty often. Usually when I hit the refresh button because the webkit-window doesn't display anything or hitting "Apply" in the Preferences.

I will provide you with some gdb backtraces and field tests.

Regards Uli

wt designer crash linux 64bit.txt

[juangburgos]

Hi Uli,

Thanks for the feedback.

I also experience this problem sometimes and sadly I believe the cause of it is Qt's webkit module. Actually that module is already deprecated in Qt5.6 and replaced by webengine which is better and less buggy.

Sadly the port from webkit to webengine is not trivial and recently I got promoted at work, which means I don't have a lot of extra time to develop WtDesigner. But I made it open source so people can contribute.

If you can or know somebody who can make the port to Qt 5.6 to fix this issue would be very welcome =)

Best,

Juan.

[cls-nebadje]

Yeah, this a recurring problem: To many nice projects but not enough time to work on them ;) I don't know if I find some spare minutes, but who knows... At least we have documented this issue.

Regards Uli

[cls-nebadje]

Hi Juan,

WtDesigner crashes continuously at this point:

[2016-Jun-16 15:17:19.489355] 25823 - [info] "WebRequest: took 146.368ms"
[2016-Jun-16 15:17:19.488581] 25823 [/ 09w6DA3vcipuPZDl] [info] "Wt: session destroyed (#sessions = 0)"
[Thread 0x7fff65ffb700 (LWP 26158) exited]
[2016-Jun-16 15:17:19.490517] 25823 [/ dWRySI7KjHGlx0bU] [info] "Wt: session destroyed (#sessions = 0)"
[2016-Jun-16 15:17:19.490573] 25823 - [info] "WServer/wthttp: Shutdown: stopping web server."
[Thread 0x7fff64b8d700 (LWP 26159) exited]
[ERROR] Could not stop Wt Server cleanly
[ERROR] Exception occurred while shutting down Wt Server in WtServerWorker::run
FATAL: exception not rethrown

This indicates that an exception is thrown but not catched when WServer is stopped. It doesn't seem that this is related to webkit. My crashes are 99% of this type and happen quite often. Somehow the server shutdown seems to be problematic. Perhaps some locking issues?

FATAL: exception not rethrown

points to another issue: You seem to catch an exception which is required for proper stack unwinding but you don't re-throw it. This happens mostly when using ellipsis-catches.

Regards Uli

[juangburgos]

I recognize the [ERROR] ... errors because those were put by me.

As you can imagine, the Wt library is not perfect and some times, when I made changes to a project, Wt wouldn´t just update the changes, so I had to make a manual shut down and restart of the Wt server in those case. That mechanism is in a function which is void MainWindow::StopWtServer() in the /src/main/mainwindow.cpp file.

What is happening is that in your case, for some reason, the Wt server does not want to shut down. I don't get those Wt problems in Windows though but what I can do for the next release is to add a try catch statement in the part of the code that shuts down the server, namely the void WtServerWorker::run()function in /src/main/wtserverworker.cpp, see if that helps out.

You can try this solution yourself without having to wait for the next release if youare compiling WtDesigner by yourself.

Best,

[cls-nebadje]

Hi Juan,

why do you wrap Wt::WServer into a QThread? Wt::WServer runs in a thread anyways - so there's no need to do so or do I miss something?

from wtserverworker.cpp:

    // configure wt server
    Wt::WServer server(m_argc, m_argv, WTHTTP_CONFIGURATION);
    server.addEntryPoint(Wt::Application, createApplication);
    // start wt server
    try
    {
        server.start();
    }
    catch (...)
    {
        qDebug() << "[ERROR] Exception occurred while starting Wt Server in WtServerWorker::run";
    }
    // eter event loop and wait
    exec();
    // stop wt server
    try
    {
        server.stop();
    }
    catch (...)
    {
        qDebug() << "[ERROR] Exception occurred while shutting down Wt Server in WtServerWorker::run";
    }

Even if I consider the server thread superfluous, the last statement should rather read something like this anyhow:

    try {
        server.stop();
    }
    /* catch WServer errors only - without masking other exceptions */
    catch (const Wt::WServer::Error &e) {
        qCritical() << "Wt::WServer::stop() in WtServerWorker::run() failed with: " << e.what();
    }

This way you don't accidentally catch unrelated exceptions and get a more detailed information about the actual error during the server shutdown. I use Wt::WServer here in pretty complex setups but never had any problems with WServer::stop(). I usually suspect my code to be the culprit until I can prove the contrary, thus in this case it is not so clear if WServer::stop() fails or Wt "is not perfect" in my humble opinion.

Regards Uli

[juangburgos]

Mainly because server.start(); is blocking. Also I thought is a good idea to have the Qt GUI in its own thread so it remains responsive regardless of anything Wt might be doing.

[cls-nebadje]

Hi Juan,

I think I can locate the bug. It's related to your thread-wrapping.

FATAL: exception not rethrown

is an error message which is system-generated and not printed by WtDesigner. It indicates that you catch an exception which you shouldn't catch. You catch it with your ellipsis on line 68 of wtserverworker.cpp. The exception itself though is caused by

        mp_WtServerWorker->quit();
        mp_WtServerWorker->terminate();

which kills the thread (instead of terminating and joining it). This causes this special (c++ runtime) exception to be raised in the thread to provide at least some limited stack-unwinding to clean up the thread which is not terminated properly. Such exceptions are not allowed to be catched without rethrowing them. But that's exactly the case here. This gets detected by the c++ runtime which in turn aborts the program by sending it the SIGABRT signal.

The reason for this is that Wt::WServer::stop() seems to take longer than 15 times 100ms = 1.5 seconds. So you interrupt the real shutdown and kill the thread.

So, what is the proper solution to this bug?

First: Catch the errors thrown by Wt::WServer::stop, let other exceptions propagate

Second: Do not kill the superfluous thread (which has been introduced exactly because stop blocks for some unknown reason - this looks like symptom fighting to me) - instead try to find out why Wt::Server::stop() takes so long (you mentioned that in previous posts). The reason for this could be some dead-locking code in your WApplication implementation - perhaps I find a quiet moment to take a look at it and give you some hints...

Generally killing threads is not a good idea, usually resulting in memory and resource leaks (file descriptors, ports, ...) - it's always worthwhile to find the real reason for some deadlocks.

Regards, Uli

[juangburgos]

THanks for the tip, just might just have resolved the issue... have you tried it?

I kill the thread because I did not want to go through the Wt code, not for lack of interest, but lack of time =(

[cls-nebadje]

Yeah, if WServer::start() takes longer you can use an asynchronous operation or threadlet, you do not need to wrap the whole start/stop sequence it into a secondary thread which apart from that does actually nothing. Same goes for WServer::stop().

Usually this is implemented by running WServer::start() in a separate thread which terminates after WServer::start() finishes. The same for WServer::stop(). Both of the threads post an event on the main loop when done - a custom event can provide you with additional information (success, exception) about the operation's result.

The Qt event loop runs free anyway, where does Wt block it in your opinion? Perhaps I can take a look at it and provide you with some hints...

Regarding the compilation: I'm currently lacking the time to do so - If I find some spare minutes I will try to set something up, but we have a release in less than a month - you know what this usually means ;) Having the compilation at hand I most probably will flood you with patches ;)

Regards Uli

[juangburgos]

I am sorry, you are right WServer::start() is not blocking, actually it might legacy code since I started this project for myself to start learning Wt, so my first versions were using WServer::run() which is indeed blocking.

I could try to improve that by removing the use of that thread but same as you, I am busy at work. I would be happy to grant you developer access to this project though, you seem to know your stuff, me I am not a software engineer, I am a control engineer which programs on his free time =) therefore I lack the undestanding that you have about threads, my knowledge is just superficial.

Let me know if you'd be interested in been a WtDesigner collaborator whenever you get less busy at work. Would be happy to add you.

Cheers,

[cls-nebadje]

Hi Juan,

that's an honour! Of course I will contribute any improvement and try to help out to make this great application better. I will let you know as soon as I have something ready or if I can do at least some consulting.

Thanks a lot! Uli

[juangburgos]

Honour's mine sir. I don´t know how it works, but I think I made the invitation already. Here is the link.

Best,

[cls-nebadje]

This should be fixed in develop-uli-crash-fixes branch.