Mem usage continuously grows

[JakeSFR]

I've noticed that ROX mem usage continuously grows over time. Starts at ~0.6% and after few hours it's 4-5% and after few more hours can even reach over 8% (of 4 GiB RAM). As you're probably aware, we're using pinboard, so ROX is always running in the background (literally and figuratively ;).

Anyway, I can reproduce it quite quickly in VirtualBox (1 GiB RAM) + Fatdog64-800pre using the following routine.

Initially, after opening HOME directory (which contains 3 dirs, 4 symlinks and 1 file, not counting hidden ones) RAM usage grows like this, according to 'top':

0. 1.9 -> 2.2

Now, if I use this one-liner to open HOME 100 times at once and then close all the windows:

for i in {1..100}; do rox -d $HOME; done; rox -D $HOME

mem usage grows again:

1. 2.2 -> 3.2

but on subsequent tries it settles:

2. 3.2 -> 3.2
3. 3.2 -> 3.3
4. 3.3 -> 3.3
5. 3.3 -> 3.3

However, if I'll increase the amount of windows from 100 to 200:

6. 3.3 -> 4.3

And to 250 (can't open 300 with only 1 GiB in VBox):

7. 4.3 -> 4.9

All this growth (1.9 -> 4.9) comes from opening only one, specific directory, multiple times.

Could you please take a look at this issue in a spare moment?

Btw, it's not a new issue, I can also reproduce it in FD-721 that comes with ROX from 2017.12. The only differences are the initial/final mem usage (slightly lower in older ROX) and that the older ROX is significantly slower when opening lots of windows at once.:)

Thanks & Cheers!

[jun7]

I think 100 windows taking only 32M RAM is normal. Do you set Options>Filer Windows>Unique windows = On? then it doesn't reproduce on my env.

Hmm, reaching 300M is not issue if you have a lot of thumbnails or huge amount of files though I think you havent that. On my env rox only uses reasonable size of RAM. Any conditions?

[JakeSFR]

I think 100 windows taking only 32M RAM is normal.

Well, the thing I don't understand is why even after closing all these windows, mem usage won't drop to the initial amount, which was 3.2, right after opening $HOME for the first time. Instead, it just stays at 4.9.

Do you set Options>Filer Windows>Unique windows = On?

No, it's off.

Hmm, reaching 300M is not issue if you have a lot of thumbnails or huge amount of files though I think you havent that.

Thumbnails are off as well. Also, I enabled 'Purge Dir Cache' and set 'Purge Time for Memory Cache' to 10, but it didn't make any difference.

On my env rox only uses reasonable size of RAM. Any conditions?

No special conditions, just using it - browsing different locations, opening files, usual stuff... How much RAM it usually uses after few hours of uptime in your case? Maybe indeed I just worry to much. :)

Anyway, tomorrow I can try running ROX for an hour or two with Valgrind's '--massif' option (as described here: http://rox.sourceforge.net/desktop/book/export/html/16.html) and send you results, if you'd like, so you could see if there's anything unusual?

[JakeSFR]

I was running ROX for ~1.5h with '--massif' option. Output attached.

ROX_massif.tar.gz

[jun7]

after closing all these windows, mem usage won't drop to the initial amount,

It is normal I think. Do others' drop? The mems are freed but not returned to OS because they are not enough size I think. I want the way to drop mem size though.

Purge Dir Cache

If you have two or more dirs having over 100,000 items, you can see it works. Normally don't check it. it is not well tested and makes memory fragment more. The size of Dir cache is small enough.

Purge Time for Memory Cache' to 10

0 is recomended and 10s is too short to work. rox keeps thumbs while on a dir even it is 0.

How much RAM it usually uses after few hours of uptime in your case?

100M - 200M. I have screen shots dir that having 500 items. So 256 256(I use 256thumbs) 3 * 500 = 100M. It is reasonable.

[JakeSFR]

after closing all these windows, mem usage won't drop to the initial amount,

It is normal I think. Do others' drop?

What others - filemanagers? I don't know, I use ROX exclusively. :)

And to clarify: I understand that after opening some location for the first time, e.g. $HOME, some things get cached in memory for future use, correct? What I fail to understand is why, if I open $HOME again, 100 times at once, mem usage grows (and won't drop back after closing) significantly - are those "things" cached before not being reused and are cached again, 100 times..?

The mems are freed but not returned to OS because they are not enough size I think. I want the way to drop mem size though.

Oh, I see! This gave me a hint and I found about this env variable: MALLOC_TRIMTHRESHOLD. I will run ROX with it set to, say, 1024 for some time and see if there's any improvement.

Purge Dir Cache

If you have two or more dirs having over 100,000 items, you can see it works.

Nope, the "biggest" dirs I have contain less than 5,000 items.

Normally don't check it. it is not well tested and makes memory fragment more. The size of Dir cache is small enough.

Ok.

Purge Time for Memory Cache' to 10

0 is recomended and 10s is too short to work. rox keeps thumbs while on a dir even it is 0.

Ok, thanks for explaining it.

How much RAM it usually uses after few hours of uptime in your case?

100M - 200M. I have screen shots dir that having 500 items. So 256 256(I use 256thumbs) 3 * 500 = 100M. It is reasonable.

Hmm, pretty comparable, although I have thumbnails disabled on daily basis (unless .DirIcons count, too).

Well, I guess I'll close it for now. I may get back to it, if I find something more substantial.

Thanks again & Cheers!

[JakeSFR]

I think I've found what's causing it in my case. Not sure if it's related to the "100 windows at once" memory spike, I doubt that, but it's related to https://github.com/jun7/rox-filer/issues/128.

I have 2 small analog clocks (AppDirs) on my pinboard. They update itself every 60 seconds and each update causes a new portion of memory being allocated by ROX. After many hours it adds up to quite considerable amount.

I've disabled them yesterday and today after 1.5h of uptime mem usage is still below 1% (31.7M).

It can be reproduced quite quickly with this script - mem usage grows ~0.4M per second:

#!/bin/sh

WORKDIR="/tmp/MEMTEST"
mkdir -p "$WORKDIR"

cat > "$WORKDIR/.DirIcon" << EOF
<svg width="512" height="512">
<circle cx="256" cy="256" r="256" />
</svg>
EOF

# MEMTEST AppDir has to be exposed in order to observe mem growth
rox -d /tmp 

while [ -e "$WORKDIR" ]; do
    touch "$WORKDIR/.DirIcon"
    rox -x "$WORKDIR"
    sleep 0.1
done

[jun7]

It is reproduced though not every time. I will take it. Thank you!

[jun7]

On my env it is rarely reproduced, so please tell me if it is not fixed.

[JakeSFR]

I tried multiple times and after a few increases shortly after start, mem usage settles down and doesn't grow anymore, like before. So, looks like it's fixed. :)

Thank you very much & Cheers!