ScudCloud chewing up CPU

[shevek]

ScudCloud Version

Paste the output for scudcloud --version below: ScudCloud 1.50 Python 3.5.2 Qt 5.5.1 PyQt 5.5.1 SIP 4.17

Distro and Desktop info

• Distribution:
• Version/Release:
• Desktop environment: No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.2 LTS Release: 16.04 Codename: xenial

Expected behavior

Uses a small amount of CPU - it's an IRC client after all

Actual behavior

Uses 75% of a broadwell CPU continuously and kills my battery.

Steps to reproduce

1. Run scudcloud.
2. Watch top

Can I run any command which will tell you WHAT is taking up all this CPU time?

[shevek]

Going through teams one by one and pressing F5 to reload them seems to have solved this temporarily. scudcloud is now using 20% not 75% CPU. That's still WAY too much, but it's better.

[raelgc]

Hi @shevek and thank you for raising this.

Of course, this is not the expected behavior (it should use much less then 20% CPU as it usually does).

It happens all the time or under any specific conditiion?

By the way, it's not a IRC client, it's a HTML engine rendering the Slack web with some desktop integration, like all Slack clients (even the official one).

[shevek]

strace -fp gives:

[pid 30595] poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=12, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=40, events=POLLIN}, {fd=51, events=POLLIN}, {fd=57, events=POLLIN}, {fd=66, events=POLLIN}, {fd=68, events=POLLIN}, {fd=94, events=POLLIN}, {fd=95, events=POLLIN}, {fd=108, events=POLLIN}, {fd=126, events=POLLIN}], 15, 8) = 0 (Timeout) [pid 30595] recvmsg(8, 0x7fff75385c80, 0) = -1 EAGAIN (Resource temporarily unavailable) [pid 30595] poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=12, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=40, events=POLLIN}, {fd=51, events=POLLIN}, {fd=57, events=POLLIN}, {fd=66, events=POLLIN}, {fd=68, events=POLLIN}, {fd=94, events=POLLIN}, {fd=95, events=POLLIN}, {fd=108, events=POLLIN}, {fd=126, events=POLLIN}], 15, 0) = 0 (Timeout) [pid 30595] recvmsg(8, 0x7fff75385c80, 0) = -1 EAGAIN (Resource temporarily unavailable)

[shevek]

fd 8 is a unix socket: scudcloud 30595 shevek 8u unix 0xffff8804190ae000 0t0 5900824 type=STREAM

[shevek]

I suspect that's the link to the X server. Is this an unchecked POLLERR?

[shevek]

Fresh restart, already consuming 40% CPU. Still file descriptor 8. Could this be to do with animated icons/badges/whatever? particularly those in channels which aren't "currently on top/displayed"?

I know it's not an IRC client, but philosophically it's an IRC client, and isn't entitled to more than 1% CPU.

[raelgc]

Philosophically ScudCloud it's a web container, rendering heavy JS usage. It's how Slack is implemented.

About this high usage: wondering if this is related to the team icon downloading not being able to finish for some reason.

You can disable (comment) these lines in wrapper.py (inside populate method):

        # Using team id to avoid invalid icon paths (Fixes #315)
        # icon_name = 'scudcloud_' + data['teams'][0]['id'] + '.jpg'
        # icon_path = os.path.join(self.window.cache_path, icon_name)
        # Download the file to use in notifications
        # file_name, headers = request.urlretrieve(data['icon'], icon_path)
        # self.icon = file_name

Another possible socket happens in __main__.py, which uses a local socket to avoid more instances running at the same time.

[shevek]

Currently poll/recvmsg calls are happening at 3-4 per second, and CPU usage is 1%. That's as expected.

When I posted earlier, I was getting 50 poll/recvmsg calls a second, and it was using 75% CPU.

I rather suspect it was the X connection. When I closed the FD in gdb, I got an X connection error.

I have all the team icons downloaded fine. This persists hours after a successful startup.

[zerthimon]

Scudcloud is chewing up craploads of CPU on my laptop. Battery is not really happy about this.

[shevek]

Observation: While strace -fp is going crazy, ltrace is singularly boring, a bit of malloc, a bit of free, but fundamentally nothing's going on, and it's spending a great deal of time idle. My guess is that someone's stuffed up a low level I/O loop and it's spinning.

[shevek]

Here is the stack of the thread which is chewing CPU:

#0  0x00007f5950485a43 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#1  0x00007f595047bc1f in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#2  0x00007f5950440911 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#3  0x00007f5950614757 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#4  0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#5  0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#6  0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#7  0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#8  0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#9  0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#10 0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#11 0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#12 0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#13 0x00007f5950614580 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#14 0x00007f595061906a in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#15 0x00007f595061a304 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#16 0x00007f594f6f8f88 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#17 0x00007f594f6f95de in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#18 0x00007f594f76910c in ?? () from /usr/lib/x86_64-linux-gnu/libQt5WebKit.so.5
#19 0x00007f59550e959f in QAbstractAnimation::setCurrentTime(int) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#20 0x00007f59550e989d in QUnifiedTimer::updateAnimationTimers(long long) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#21 0x00007f59550ea4ec in QAnimationDriver::advanceAnimation(long long) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#22 0x00007f5955320bb3 in QObject::event(QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#23 0x00007f594799105c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#24 0x00007f5947996516 in QApplication::notify(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#25 0x00007f5947181f3e in ?? () from /usr/lib/python3/dist-packages/PyQt5/QtWidgets.cpython-35m-x86_64-linux-gnu.so
#26 0x00007f59552f138b in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#27 0x00007f59553465ed in QTimerInfoList::activateTimers() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#28 0x00007f5955346b29 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#29 0x00007f5953fc6197 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#30 0x00007f5953fc63f0 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#31 0x00007f5953fc649c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#32 0x00007f59553477cf in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#33 0x00007f59552eeb4a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#34 0x00007f59552f6bec in QCoreApplication::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#35 0x00007f59471450bb in ?? () from /usr/lib/python3/dist-packages/PyQt5/QtWidgets.cpython-35m-x86_64-linux-gnu.so
#36 0x00000000004e9b9f in PyCFunction_Call ()
#37 0x0000000000524414 in PyEval_EvalFrameEx ()
#38 0x0000000000528814 in PyEval_EvalFrameEx ()
#39 0x000000000052d2e3 in ?? ()
#40 0x000000000052dfdf in PyEval_EvalCode ()
#41 0x00000000005fd2c2 in ?? ()
#42 0x00000000005ff76a in PyRun_FileExFlags ()
#43 0x00000000005ff95c in PyRun_SimpleFileExFlags ()
#44 0x000000000063e7d6 in Py_Main ()
#45 0x00000000004cfe41 in main ()

Derived from here: https://stackoverflow.com/questions/23838498/how-to-diagnose-a-python-process-chewing-cpu-in-linux#23852420

Now, if those timers are coming either from the expiry of the select() or over the socket from X, that would be a start. Now how about a minimum time between timer firings of say 10ms?

[shevek]

... and it is the damn animations, which was my guess earlier. We need to disable animations not in visible panes. Anything except the absolute current window should NOT be animating.

[shevek]

Going into every slack window and switching the 'channel' to DM to slackbot (which never posts animations) makes the CPU usage drop to 1%, which is where it should be.

[shevek]

Further experimentation: Yes, it's almost definitely the animations. A minimum granularity of a shared 100ms or even 200ms timer should be just fine. This is still effectively IRC, not Netflix. :-) No background work needs to happen at more than 5Hz?

[shevek]

interestingly, QT even chews CPU if the animation is scrolled offscreen, as long as it's in the foreground channel.

[raelgc]

@shevek One time I tried to list all Timeouts in the JavaScript side (i.e., Slack), and I saw more then 10. In the Scudcloud side, there are, of course, the QT UI threads, and one additional thread that will keep just sending a tick sign to Slack to keep the user updated.

QT chews CPU too if you try to expand a code snippet in Slack.

[shevek]

Most web browsers don't run timeouts for offscreen animations, and limit the granularity of setTimeout() in JavaScript for this very reason. This really is about animations.

[raelgc]

I'm not saying timeouts are related to offscreen animations. I'm just sharing how much resource hog is Slack.

[raelgc]

The next upgrade in Qt for web-container is to move from webkit component (QWebView, which will be marked as deprecated) to Chromium component (QWebEngineView).

I'd prefer if they upgrade the webkit version, but it's not my choice.

For sure, it'll use more memory, but at least, it'll use less CPU for heavy JS and animations.

[immerrr]

@raelgc there's a rather active backport of webkit that supports fresh qt at https://github.com/annulen/webkit, but I have never tried to build it