SystemsGenetics / KINC

Knowledge Independent Network Construction
MIT License
11 stars 4 forks source link

Examine the performance impact of using Qt #75

Closed bentsherman closed 5 years ago

bentsherman commented 5 years ago

I just fixed a problem that happens in RMT when the cmx is really large -- the pruned matrix was roughly 25000 x 25000 and apparently trying to allocate a QVector with that many elements caused an exception. It wasn't enough elements to cause an integer overflow, and it didn't require more memory than what was available, but when I replaced QVector with std::vector, it worked. So it seems that QVector had some kind of limitation that std::vector did not.

But this issue has sort of catalyzed a deeper concern that I've had about using Qt with high-performance code. Don't get me wrong, I think Qt has been extremely helpful to us in developing a complex application with all of its features. And Qt even claims to be "high-performance". But "high-performance" is a relative term when also considering ease-of-use, portability, and GUIs, as Qt does.

So while I am hopeful that we won't have to choose between Qt and maximum performance, I think it would be worthwhile in the long-term to determine if (and to what extent) the scope of Qt should be restricted in KINC code (and ACE code by extension). In other words, are there certain functions (or even entire classes) in which Qt classes should not be used at all? Obviously anything GUI-related will pretty much be all Qt, but a lot of the analytics internals, the portions of code that get called a lot in a large-scale setting, may benefit from using C++ STL instead of QTL, for example.

Or maybe it doesn't matter. The point is that right now we don't really know, and if we are marketing ACE and KINC as high-performance tools then I think we need to be able to argue that our use of Qt does not hurt performance. And if it does, then we need to curtail our use of Qt where applicable.

4ctrl-alt-del commented 5 years ago

I highly doubt the bug you discovered in QVector is a feature. I would argue it is not a limitation of QVector but a simple overflow bug you found in their library.

I have no concern in regards to Qt and high-performance. If it can be proven otherwise, obviously I would change that stance. Some thing are faster in Qt compared to the standard library and others are faster the other way around. qobject_cast, for example, is usually ten times faster then dynamic_cast. The biggest reason however I have no concern is Qt does a very good job separating their GUI code from their core code(AKA qtcore). The core library has one singular goal for Qt; performance. Another good example is until C++11 Qt blew the standard library out of the water with their COW(Copy On Write) practice. Even now Qt is often much more efficient when making constant copies of objects like strings or container classes.

As I said, restricting the core library of ACE and the analytics of KINC to qt core should be satisfactory.

Beyond that, I am fairly certain it does not matter, and I did some basic tests to back that up:

QVector vs. std::vector I called push_back one million times for each one. QVector: 342 ms std::vector: 192 ms

QList vs. std::list I called push_back one million times for each one. QVector: 255 ms std::list: 937 ms

QMap vs. std::map I called insert one million times for each one, using unique keys. QMap: 5044 ms std::map: 11487 ms

As you can see, some thing are faster in different libraries. I would argue in those three tests combined, qt won in performance because it got 2 out of 3 and the list container was dramatically faster instead of just twice as fast.

Also, I specifically use QMap in a very performance sensitive bottleneck( the MPI Master node ) in ACE and if I changed that to std::map it would dramatically hurt ACE performance.

In closing, I think this is a non-issue.

4ctrl-alt-del commented 5 years ago

I realized I did those timing tests with debugging enabled. Here are the same tests with debugging disabled:

QVector: 82 ms std::vector: 83 ms

QList: 109 ms std::list: 373 ms

QMap: 2827 ms std::map: 2838 ms

Thanks to QList Qt still wins, everything else is literally the same. So this still supports my lack of concern over using qt core classes.

bentsherman commented 5 years ago

Awesome, well that makes me feel a lot better about QTL at least. Hopefully my problem was just a bug like you said.