Exhausting memory

[vitaaliy]

From time to time Windows Task Monitor shows again saw-like RAM consumption... Then amount of used memory gradually increases up to the physical limit: 8 Gb... blocking any computer activity. Several Firefox processes run in parallel, size of one eats the main part of the memory available. Bookmarks: 84082 Favicons to fetch: 83992 (99.9%) Folders: 15677 Separators: 3 Oddities: 0 Closing and reopening of FF helps... for several hours. With BSP2 disabled all works normally.

[andreygursky]

Hi @vitaaliy, this sounds like https://github.com/aaFn/Bookmark-search-plus-2/issues/49.

While it cannot be really solved now, @aaFn has kindly implemented a workaround. Please try to switch "Favicon fetching" to "Pause" in BSP2 preferences.

[vitaaliy]

Thank you, Andrey, but I switched favicon fetching to "Pause" long, long ago. Now I think about disabling favicon fetching completely and forget about them as of horrible dream. Really do they deserve such sacrifices? I am grateful to aaFn for this BSP2 addon - I cannot imagine how I would work with my bulky bookmarks collection without it. Comparing to the main BSP2 effect - favicons are like bells and whistles at its side...

[aaFn]

Hello @vitaaliy , strange, can you check again on the options page and tell me ? With favicon fetching set to pause, there shouldn't be any memory saw problem, The only difference with disabling it are:

• that what is already in memory is kept, while with "disabled", it is forgotten
• you can refresh manually a favicon and it will work, as it is a voluntary action.

Let me know, if you confirm it is set, I might have a bug lying around ...

[vitaaliy]

Okay @aaFn - I have just enabled BSP2, took a screenshot of BSP2 options and rerun FF. Now it is all quite and memory consumption is about 3 Gb. Probably the devil will show himself tomorrow. I will keep you informed.

[vitaaliy]

In the afternoon the saw started... , but then disappeared, and the memory level now is 5.8 Gb.

[aaFn]

Ok, were you doing anything to bookmarks at that time, or do have you another add-on related to bookmarks installed ? Or did you restart BSP2 in any way ?

Asking because there is no reason that BSP2 does anything by itself in the background, it should only be because notifications are sent to it by FF (like modified bookmark), either originated by the user or by another add-on acting on bookmarks.

[vitaaliy]

No another add-on rivalry to BSP2 ;). Yessss... when new FF version arrives I try to find there such intrinsic functionality - but fail each time. All my bookmark activity is in adding new, deleting, moving and sorting. Sometimes I get some BM subtree lost and planted somewhere - God knows where - and at such moments I ask BSP2 to find this escapee - it finds it always :) - and I move it to its rightful place. Now I see no saw activity. Memory used is stable about 5 Gb.

[aaFn]

Ok, let me know if it starts again, I re-checked in the code and so far saw nothing obvious that would start anything like that by itself.

[vitaaliy]

O.K. I will complain here if any trouble happens :)

[vitaaliy]

Longly awaited trouble showed itself. BSP2 was enabled for about 10 hours. BM activity was negligible if any. Most part of the time FF was left to itself. No memory saw. But the amount of memory used gradually increased until it almost exhausted all RAM available. FF has about 10 processes, and one of them used about 6 Gb. Processor usage was about 1%. Certain HDU activity. At this state reaction to the keyboard or mouse was very reluctant and delayed. With considerable efforts, I disabled BSP2 and reloaded FF. It displayed the diagnostic message about a breakdown. Now it works normally, BSP2 disabled (alas!), memory used is about 4 Gb. No delays, no memory saw... Of course, I am not sure that the cause of these misbehavior lies in the BSP2. And besides my drawback is the huge amount of opened tabs... maybe more than 200... Any hints?

[aaFn]

Hello @vitaaliy , I made an experiment on my side with about 30000 bookmarks and fetching favicons intensively. That lasted for 6 hours, and at the end of the 6 hours, one process was around 2 GB in FF.

I opened a tab with the about:memory URL and this gave me the following results:

• The Main Process pid was around 2 GB -> this the one !
• One of the Web Extension pid was 500 MB -> I could determine this was BSP2, for both the sidebar and the background, and all looked normal in there.

Details of "Main process" are showing a lot of strings and classes which are taking a lot of space in the Main process; plus a big heap. See below:

2,211,922,900 B (100.0%) -- explicit
├──1,509,569,568 B (68.25%) -- js-non-window
│  ├──1,402,184,144 B (63.39%) -- zones
│  │  ├──1,271,322,704 B (57.48%) -- zone(0x238ac952000)
│  │  │  ├────632,475,320 B (28.59%) ++ strings
│  │  │  ├────557,393,504 B (25.20%) -- realm([System Principal], DevTools (Module loader))
│  │  │  │    ├──556,582,416 B (25.16%) -- classes

├────502,673,040 B (22.73%) ── heap-unclassified

While things seem normal for BSP2, given it has around 30000 objects ..

WebExtensions (pid 208)
Explicit Allocations

534,093,824 B (100.0%) -- explicit
├──428,383,568 B (80.21%) -- window-objects
│  ├──216,611,936 B (40.56%) -- top(moz-extension://cf381681-36b0-41e7-bd9c-226aa4373319/sidebar/panel.html, id=4294967315)
│  │  ├──211,104,832 B (39.53%) -- active/window(moz-extension://cf381681-36b0-41e7-bd9c-226aa4373319/sidebar/panel.html)
│  │  │  ├──204,303,280 B (38.25%) ++ dom
│  │  │  ├────4,342,320 B (00.81%) ++ js-realm(moz-extension://cf381681-36b0-41e7-bd9c-226aa4373319/sidebar/panel.html)
│  │  │  ├────2,447,408 B (00.46%) ++ layout
│  │  │  ├───────11,360 B (00.00%) ── property-tables
│  │  │  └──────────464 B (00.00%) ── bindings
│  │  └────5,507,104 B (01.03%) ++ js-zone(0x2003b13b000)
│  ├──137,241,328 B (25.70%) -- top(moz-extension://cf381681-36b0-41e7-bd9c-226aa4373319/_generated_background_page.html, id=4294967312)
│  │  ├──128,198,752 B (24.00%) -- js-zone(0x20039a12000)
│  │  │  ├──124,343,112 B (23.28%) ++ strings
│  │  │  ├────2,854,952 B (00.53%) ++ shapes

So I am concluding that there is a form of "memory leak" in the main FF process :-(

It looks I cannot do much about it, apart trying to avoid situations which are creating those memory leaks ...

• In your case, you told me above that favicon fetching was paused
• You also told me you had a lot of tabs
• So I guess you are switching very frequently from tab to tab, as you navigate
• Switching from tab to tab triggers an event in BSP2, so as to try to catch favicons from tabs to accelerate things, exactly like the Native bookmark sidebar is doing
• This triggers a browser.bookmarks.search(), and then if the URI in the tab is not a data: ready made URI, and we are not in pause favicon fetching, a favicon fetch.
• I am presuming the browser.bookmarks.search() could be at origin of the memory leak ...

So in 2.0.41 I am changing the code so as to not do browser.bookmarks.search() if we are in pause favicon fetching and if the tab does not hold a data: ready made URI (data: is very rare).

Should be out tomorrow. Hope this will help ...

[aaFn]

Meanwhile, if you could yourself do an about:memory when you are in a situation with lots of memory eaten, that would be interesting to look at the result ...

[vitaaliy]

Last two days RAM never exhausted completely... Now the system eats 6.45 Gb. No exaggerated saw seen. FF processes eat memory considerably, but nothing outstanding: 200 - 600 Mb. I attach about:memory report here. But before that, when things were really tough one of the FF processes really ate outstanding memory amount: several Gigs... memory-report.json.gz Thank you for your attention and labor :)

[aaFn]

Indeed, all looks good on the BSP2 side in your capture, I see around 600 MB, with good proportion of DOM, which is normal given the number of bookmarks you have. And Main process is a little bit more, but nothing exagerated in there as you say ..

We would need to see, when you have a FF eating a lot, if it is like the case I could provoke above. If that happens again and you are able to make an about:memory capture, I would be quite interested.

[vitaaliy]

Here it came... Gross memory saw, Then almost all memory been eaten. I managed to take two pictures: and: . But comp's activity approached to almost zero... I planned to fix "about:memory" (at that moment the heaviest FF process has grown up to 6 Gigs probably, but all my actions were almost frozen, and I failed... Memory consumption became swiftly lower... and all FF processes were again looking almost pretty - I even decided not to take the last measurement. Now all is again calm as still Pacific. Total memory expenditure - about 3.8 Gb. No saw. Such is life...

[aaFn]

Ok .. too bad .. next time will be better maybe ...

[aaFn]

@vitaaliy, 2.0.41 is out with the modification of not searching for bookmarks of a favicon refresh from a data: URI, when you switch from tab to tab and the pause favicon fetching option is activated, in hope to not fall in the case of "memory leak" I can observe in the Main process.

Let me know if that changes anything to the memory problem.

Thank you, aaFn.

[vitaaliy]

Thank you, @aaFn, I will try 2.0.41 this evening. But first my fairy tale. In the morning I had BSP2 enabled. Then I noticed memory push up, comp freezing... Very bad. Then I disabled BSP2, shut all FF processes one by one - manually - hoping it will take less time and then started again FF (BSP2 still disabled). It started to function normally - nothing outstanding... But half an hour ago I noticed huge memory saw: . One of the FF processes swelled up to 1.6 Gb - as it was before when BSP2 was enabled. Though the memory level consumed was moderate. At the end of this performance, the memory level stabilized at the level of about 3 Gb. What is moral? Probably the saw effect is due not to the BSP2 (or not only to the BSP2) but to intrinsic mechanisms of FF?

O.K. I have just made clean BSP2 2.0.41 installation: removed previous version reloaded FF and installed the new one. Comp did not exploded, no smoke or strange noises... Something is going on - as memory level is not stable...

[aaFn]

Hello @vitaaliy , thank you for the feed back. Yes, if the phenomenon still occurs when BSP2 is disabled, the origin should lie somewhere else, and maybe BSP2 would be an amplifier. Hoping the modification I made in 2.0.41 then would stop the amplification.

When looking at your about:memory output, I noticed other threads for other web pages and maybe extensions which are taking much space. The origin might lie there.

For example, there is this one https://addons.mozilla.org/en-US/firefox/addon/fess-google-bookmark-extens/ which could interact with BSP2 in some way if they are both active at the same time, because using the same API.

However, in your dump, BSP2 was the only active extension, so that's not it I see that a few web pages PID's concentrate a lot of memory, though, not sure what they are.

=> When examining a "memory dump" that you take, look at the "Web Content (pid xxx)" ones, and look at the 3rd line in "Explicit Allocations", that should tell you which one is taking a lot of memory.

Elsei, if the problem lies with FF internal mechanics, then there is not much we can do :-(

[vitaaliy]

Hi, @aaFn, I think it would be reasonable to watch the situation for several days - too many factors influencing the overall behavior - we should see how the system will behave in the long run. Right away it is all calm and in perfect condition. I have the own FF bookmarks on, BSP2 enabled and ready for help when needed :)

[aaFn]

Ok, need to be patient to catch the beast .. :-)

[vitaaliy]

Hi @aaFn … Concerning our beast. Three days already it tries to put our guard to sleep. No suspicious activity from the BSP2 side is displayed. All FF processes remain similar in size, no those catastrophic N-Gbytes outlaws. No motives for BSP2 disabling. Its functionality is OK. Looks like your current version is good :)

[aaFn]

Excellent, thanks for taking the time to feed back :-) I will leave the issue open 1 more week just in case it returns, and if no sign, I'll close.

Thank you, aaFn.

[vitaaliy]

Hmmmm... Hidden beast reminded of himself, a terrible roar sounded and a disgusting stench dissolved in the air ... The level of the memory used was approaching the limit, among FF processes again appeared 1-2 processes with memory > than 1 Gb. Oscillogram of the memory usage took specific form with rectangular ups and downs, and the most unpleasant thing - comp's activity was slowing down, reaction to the mouse manipulations has become extremely reluctant and delayed. I disabled BPS2 and reloaded the FF - all became calm and smooth. I understand that this poetic description gives almost nothing in finding the cause of these effects, but such is life... :(

[aaFn]

Hello @vitaaliy , indeed what we would need is an about:memory output at that moment, if at all possible, to see if something appears .. although I begin to have the strong suspicion that it would be the "Main process" and if that is the case, that won't tell much :-(

Anyway, sorry for that, in the absence of anything else, there's not much more to do than disabling I guess. aaFn

[vitaaliy]

Several hours there was no deadly tight situation. I decided to show you the typical pictures. memory-report2.json.gz

[aaFn]

Thanks, I see that there are plenty of objects not freed by the GC in BSP2 .. always the same problem since I cannot call the Garbage Collector myself from the code to force it to free unused objects :-(

A few questions as I see some things doubled in the memory report:

• Are you using the "Dual save" option ? If so, I would advise to disable it.
• Can you copy / paste here the contents of your Statistics box in the Options page ?
• Can you activate the traces option in the options panel, copy here the contents of the trace textbox which appears at top of the sidebar (and then deactivate traces, no need to keep them) ?

This is to see if you do not still have a 16x16 migration active in background (useful to reduce storage, but activating frequent saves to local storage which appears to be the majority of unused objects still in memory before being discarded by the GC).

On my side, I will try a simplification in 2.0.43 in the save to local storage code, to try to avoid the dual objects I see in the memory report.

Thank you, aaFn.

[vitaaliy]

1. Hmmm... Incidentally, I found that "Favicon fetching" was "Active" from some unknown moment in the past/ I switched it to "Pause" - as it was earlier. Maybe it is the main cause of the recent awful behavior... While I look now on "Performance" oscillogram... - it looks quite decent, without the saw and that drastic ups and downs... I ought to watch this for some time.

2. "Dual save" was and is disabled

3. Here is Statistics: Bookmarks: 84444 Favicons to fetch: 48142 (57.0%) Folders: 15743 Separators: 3 Oddities: 0

4. At the options end is the key: "Re-trigger 16x16 favicon migration" - I never touched it.

5. Traces: Load local store duration: 102 ms Background load local store duration: 4703 ms Background bypassed FF API for tree load Background tree load duration: 57 ms Background tree build duration: 33 ms Background save duration: 5699 ms Get tree duration: 0 ms PlatformOs: win Setting Windows variations structureVersion: -img16-bnlist-spfldr disableFavicons_option: false pauseFavicons_option: true Display duration: 6251 ms Total duration: 6353 ms fontFamily = '"Segoe UI"' fontSize = '12px' Stats:

   Bookmarks: 84444 Favicons to fetch: 48142 Folders: 15743 Separators: 3 Oddities: 0

   BSP2 version: 2.0.42 Favicon display duration: 20908 ms

[vitaaliy]

The latest appearance of the beast - in pictures - memory-report3.json.gz

[aaFn]

Hello @vitaaliy , ok, yes there are still more than 48000 favicons to fetch, so that was surely doing things in the background if "pause" was not on. With pause on, it shouldn't be doing things anymore.

Now, inside the memory report above, I only see the FF main process with roughly 535 MB RAM allocated, nothing else. Are you sure the report is complete ?

[vitaaliy]

Of course, I suspected that it is not OK with it. Something I didn't do properly. I had not or just forgot the concise instructions - just called about:memory and after some time made a copy. How it should be done correctly?

[aaFn]

Inside the about:memory tab, clock on the Measure and save... button. It will create the full file immediately.

[vitaaliy]

Very strange... it was exactly what I had done. Okay. I will enable BSP2 right away and try to repeat that experiment...

[vitaaliy]

It was very tiresome to take "about:memory" - FF was "Not Responding", dimmed and obeyed no commands. At last, I got: "Saved memory reports to G:\downloads\memory-report.json.gz (2018-12-01T21:12:18.041Z)". memory-report.json.gz

[aaFn]

Thank you. Alas, same problem .. As a comparison, this file is only 76 KB, and the previous one, also incomplete, was 92 KB ... The complete one above which was holding all PID's (memory-report2.json.gz) was 564 KB

Not sure what is going wrong here, but the memory report is only doing "Main process".

How long does it take to get the memory all used problem ? Maybe the memory report cannot complete when all memory is used, so it could be useful to do the memory report when "mid of life", halfway between start and usual time when things become hectic ...

I just published 2.0.43 with a somewhat simplified bookmark tree save procedure, in the hope to ease the Garbage Collector work .. if you can try with that one ... I don't have much more clue at this time. (of course, if you can verify one more time that favicon fecthing is still paused ..)

[aaFn]

By the way, just thinking a second time about it, when you measure and save a memory report, I suggest that you first press on the Minimize memory usage button and wait for its completion, before pressing on the Measure and save.. one.

This should clean whatever is not anymore used, and include in the memory report only what is really used.

[vitaaliy]

OK. I removed 2.0.42 version and installed 2.0.43. Now BSP2 is enabled though currently I use the standard FF bookmarks.

"How long does it take to get the memory all used problem? Maybe the memory report cannot complete when all memory is used, so it could be useful to do the memory report when "mid of life", halfway between start and usual time when things become hectic ..."

• Now the situation with memory is ideal - no changes, no saw, the level is absolutely stable - 4.31 Gb. FF processes take 200 - 500 Mb. My previous experiment took an indefinite time to achieve the critical state - several hours, 1- 2 days - God knows... Each hunting The Beast was specific. Your consideration about lack of memory to create the complete report sounds realistic. Next time when the Beast shows his muzzle halfway - I will try to shoot him and send you the resulting memory report. Favicon fetching is paused - confirmed ;).

Right away I tried as an experiment memory minimization - Memory minimization completed (2018-12-02T10:27:32.439Z). And then - measure and save... You are right - the file obtained has 806 Mb - with an absolutely placid situation and a huge heap of free memory. memory-report.json4.gz

[aaFn]

Ok, yes, that memory report is complete, and BSP2 is only the 3rd consumer in memory size inside it. All looks good for now.

[aaFn]

Hello @vitaaliy , in waiting for more sign from the beast, I will close the issue, and we will reopen it if it comes back.

Thank you, aaFn.