search

JGRennison / retcon

A cross-platform Twitter client using wxWidgets.
GNU General Public License v2.0
8 stars 2 forks source link

Retcon eats all CPU if not run for a while #6

Open philpem opened 7 years ago

philpem commented 7 years ago

Steps to reproduce:

  1. Don't run Retcon for a few weeks.
  2. Run Retcon. Also run Htop (or similar) to monitor RAM usage.
  3. Wait for a few minutes.
  4. Watch the RAM usage gauge slowly creep up. Eventually the system becomes unusable due to excessive disk swapping, and Retcon has to be kill -9'ed to recover the system.

This seems to be some kind of cache cleanup which is done in the background (as Retcon is fully functional up until the RAM runs out) and causes a ton of RAM to be consumed. Perhaps this cleanup could be done in stages to reduce the system load?

If this is what it is -- a progress indicator somewhere on the main screen might be nice too... "Cache cleanup, 0-100% done"

JGRennison commented 7 years ago

There are various database cleanups and cache operations which are done periodically, however when testing I've not had them use that much time or memory, even with quite large and old databases.

One thing that I did notice when testing though is that when loading an old database a rather excessive number of network connections are opened in parallel. Limiting the number of simultaneously open connections reduced the memory use quite significantly. I've pushed a commit 4883bfcd which does this.

I'll carry on looking and doing some more tests, to see if I can find anything else.

JGRennison commented 7 years ago

I found a memory corruption issue (fixed in dfc2e658) which may be related to this. Can you let me know if you still have the problem with this applied?

philpem commented 7 years ago

I will indeed! I'll leave my laptop off Twitter for a few days and reactivate the "really busy" account. That should give Retcon something to grind through.