search

Ketok4321 / speedtest

A graphical librespeed client written using gtk4 + libadwaita
GNU General Public License v3.0
31 stars 16 forks source link

Excessive CPU and RAM usage #29

Open yochananmarqos opened 9 months ago

yochananmarqos commented 9 months ago

This is a fine GUI, however as the title says, it's very intensive. I'm using 1.2.1 via my speedtest-librespeed AUR package on Manjaro GNOME (unstable branch).

The speedtest succeeds, however the terminal output seems to complain while it runs:

❯ speedtest-librespeed
Trying to fetch servers...
Exception ignored in: <coroutine object LibrespeedBackend.get_servers.<locals>.check_server at 0x7f5ae4d78840>
Traceback (most recent call last):
  File "/usr/share/speedtest/speedtest/backends/librespeed.py", line 37, in check_server
    async with session.get(server.pingURL, timeout=aiohttp.ClientTimeout(total=2.0)) as _:
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 574, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 911, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 1204, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1147, in _create_connection_transport
    transport.close()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 112, in close
    self._ssl_protocol._start_shutdown()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 617, in _start_shutdown
    self._abort()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 667, in _abort
    self._transport.abort()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 821, in abort
    self._force_close(None)
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 891, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object LibrespeedBackend.get_servers.<locals>.check_server at 0x7f5ae4d78940>
Traceback (most recent call last):
  File "/usr/share/speedtest/speedtest/backends/librespeed.py", line 37, in check_server
    async with session.get(server.pingURL, timeout=aiohttp.ClientTimeout(total=2.0)) as _:
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 574, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 911, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 1204, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1147, in _create_connection_transport
    transport.close()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 112, in close
    self._ssl_protocol._start_shutdown()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 617, in _start_shutdown
    self._abort()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 667, in _abort
    self._transport.abort()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 821, in abort
    self._force_close(None)
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 891, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object LibrespeedBackend.get_servers.<locals>.check_server at 0x7f5ae4d79440>
Traceback (most recent call last):
  File "/usr/share/speedtest/speedtest/backends/librespeed.py", line 37, in check_server
    async with session.get(server.pingURL, timeout=aiohttp.ClientTimeout(total=2.0)) as _:
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 574, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 911, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 1204, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1147, in _create_connection_transport
    transport.close()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 112, in close
    self._ssl_protocol._start_shutdown()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 617, in _start_shutdown
    self._abort()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 667, in _abort
    self._transport.abort()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 821, in abort
    self._force_close(None)
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 891, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object LibrespeedBackend.get_servers.<locals>.check_server at 0x7f5ae4d79d40>
Traceback (most recent call last):
  File "/usr/share/speedtest/speedtest/backends/librespeed.py", line 37, in check_server
    async with session.get(server.pingURL, timeout=aiohttp.ClientTimeout(total=2.0)) as _:
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/client.py", line 574, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 911, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 1204, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1147, in _create_connection_transport
    transport.close()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 112, in close
    self._ssl_protocol._start_shutdown()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 617, in _start_shutdown
    self._abort()
  File "/usr/lib/python3.11/asyncio/sslproto.py", line 667, in _abort
    self._transport.abort()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 821, in abort
    self._force_close(None)
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 891, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Task was destroyed but it is pending!
task: <Task pending name='Task-62' coro=<TCPConnector._resolve_host() running at /usr/lib/python3.11/site-packages/aiohttp/connector.py:884> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.11/asyncio/futures.py:387, Task.task_wakeup()]> cb=[shield.<locals>._inner_done_callback() at /usr/lib/python3.11/asyncio/tasks.py:891]>
nekohayo commented 3 months ago

The tracebacks do not happen during operation, but after you close the app with the "X" button, as far as I can tell.

My total wild-guess hunch is that probably your GUI is spamming the graphics stack by refreshing way too often (the dev could probably throttle the labels + charts updates? If they update every milisecond or so, that would be very wasteful), and/or spamming glib signals, or something like that. I haven't looked at any code, this is just a guess.

More objectively, the developer can confirm whether that hunch of mine is true or not based on what we can see from profiling with Sysprof 46 (devs, let me know if you need the sysprof capture file), as shown below.

Flame graph of most frequent/expensive function calls:

Speedtest sysprof - flame graph

Graphics marks (solid blocks like that seem to lead credence to my hypothesis that you are spamming the graphics stack):

Speedtest sysprof - graphics marks

yochananmarqos commented 3 months ago

Ah, you're right about the tracebacks happening after closing the application. Sorry, I didn't realize that's when it actually occurred.

Everything else you said is way over my head. I'm not "doing" anything other than running the program.

nekohayo commented 3 months ago

With "you" I meant the developer generally.

Another quick hunch on my theory above: whatever is happening, it's causing about 62 thousand function calls in the app over the span of 29.6 seconds. That's a bit over 2000 function calls per second, which seems very wasteful, I don't see why you'd need more than, say, 60 calls per second on a 60 Hz monitor based on vsync. Even on high refresh rate monitors at 240hz, it would still be ten times less than what is happening now.

nekohayo commented 3 months ago

Just to be sure, as I narrowed down the tracebacks to occurring only outside of the flatpak version and causing incorrect measurements (see issue #37), I have retested the performance issue here with the flatpak version, and it still happens; 100% single-core CPU usage, so the function calls spammage performance issue itself is apparently not caused by the tracebacks (and it happens even before the upload test occurs anyway).