[Bug] v4.0 made scrolling very slow

[DarthGandalf]

Abstract

Since several days ago (I guess, since v4.0) the panel became very slow for me to scroll through. I have huge number of tabs, and after moving the scrollbar it just shows gray for several seconds.

Steps to reproduce

1. Any idea how to easily open hundreds of tabs on a clean profile? I'm not going to do that manually
2. Grab the scrollbar using the mouse
3. Move the mouse up or down, dragging the scrollbar
4. See gray rectangle in the place where tabs are supposed to be

Expected result

It should be fast; before 4.0 it was much faster - there was no delay between me scrolling and me seeing the tabs.

Actual result

veeeeeryyyyy sloooooooooooowwwww

Environment

• Platform (OS): Linux
• Version of Firefox: both 115.8.0esr and 122.0.1
• Version (or revision) of Tree Style Tab: 4.0.1

[chunderbolt]

Any idea how to easily open hundreds of tabs on a clean profile?

If you don't care about them being tree-like, you can hold CTRL-T to open a load of new ones.

Alternatively, maybe try bookmarking your current session, exporting it to the new one, and using "Open All Bookmarks"/"Open All as a Tree" on it.

[piroor]

Any idea how to easily open hundreds of tabs on a clean profile? I'm not going to do that manually

1. Giving "bookmarks" permission to TST (TST options => context menu =>allow to read and create bookmarks)
2. Right-click on a tab and choose "Select All Tabs".
3. Right-click on a tab and choose "Bookmark Selected Tabs".
4. Confirm that.

Then you'll get a new bookmark folder with bookmarks containing tree structure information. On a new profile, after you grant the bookmarks permission for TST and right-click on the imported bookmark folder, you'll be able to open them with tree structure.

[piroor]

@DarthGandalf How many tabs do you open? Could you collect a performance profile? https://github.com/piroor/treestyletab/wiki/How-to-inspect-tree-of-tabs#how-to-collect-a-performance-profile

[ssergiienko]

@DarthGandalf How many tabs do you open? Could you collect a performance profile? https://github.com/piroor/treestyletab/wiki/How-to-inspect-tree-of-tabs#how-to-collect-a-performance-profile

I have the same problem and I use 860 tabs at the moment. I had to downgrade back to <4 version because of this problem + all other problems from 4+ (e.g. 3475) Maybe a bit later I'll be able to do perf profile

[DarthGandalf]

Firefox 2024-03-12 17.02 profile.json.gz

Array.from(TabsStore.windows.values())[0].export(true).length says 5673.

This is from 115.8.0esr profile, can do the 122.0.1 later today.

[piroor]

Thanks! After more research I've found a bottleneck around detection of overflow state of tab labels. The profile at https://github.com/piroor/treestyletab/issues/3477#issuecomment-1992154657 also says it is the bottleneck on the case. I've released 4.0.2 with a fix for that, please wait it is published.

[gtimbo]

This seems to make things faster, but now, when scrolling up, I get a line painted across the bottom of the plugin, which only disappears when I scroll back to the very bottom.

[DarthGandalf]

It did become faster. But not as fast as before.

Specifically, if I hold the mouse button on a scrollbar much below the current position, it is now staggering: it scrolls by one screen, it has a gray part instead of tabs, fills the gray part of the screen with tabs, then proceeds to scroll by one more screen, then visibly filling the gray part with tabs, etc. Every time it pauses by 0.2 seconds or so to do the rendering (I guess I should say adding/removing tabs from DOM).

[DarthGandalf]

Firefox 2024-03-14 11.13 profile.json.gz

Profile for v4.0.3 on 115.8.0esr

[piroor]

@gtimbo It is intentionally shown to indicate there are more tabs after the visible area. However it should appear at the bottom of the view port and something wrong on your environment. Do you use any user style sheet to customize? Could you share exported config to more inspection?

[piroor]

@DarthGandalf Thanks a lot! I've also collected performance profile with ESR115 and compared with yours. They both cases say style computation is the bottleneck. So simply creation of large number DOM elements looks to take more time on your case than mine. I think it may be due to the difference of the basic performance of the PC.

Hmm, to solve this bottleneck we looks to need reusing detached tab elements...

[piroor]

I've introduced changes to reduce generating of HTML elements with reusing them. On my environment x1.8 tabs are rendered while same time so the performance become better a little, but it does not reduce style computations, thus the optimization may not improve the performance well. Could you try the development build? https://github.com/piroor/treestyletab?tab=readme-ov-file#development-builds

[DarthGandalf]

I don't see much difference in performance there.

[piroor]

Hmm, then could you try setting a secret option outOfSecreenTabsRenderingPages introduced by the commit 4940e1a to very large number (ex. 99999) ? It is configurable under TST options => Development => All Configs. It will render all tabs statically like TST 3.x so no rebuilding while scrolling will happen.

But there is a trade off: building very large number tabs takes more time when you close and reopen the sidebar, and you may see other performance problems caused from such large number tab elements. Current virtual scrolling is a design preferring total performance except too frequent re-rendering of tabs like this case.

[DarthGandalf]

4.0.3 on my 122.0.1 profile (and different computer) looks pretty good - scrolling by holding the mouse button is fast, and scrolling by dragging the bar is not as fast, but acceptable. Looks like I can't install the dev build of extension on this Firefox version though.

I will try outOfSecreenTabsRenderingPages (is "secreen" a typo?) on my ESR profile tomorrow.

[piroor]

Oops I used typo-ed name... with the commit e0c5ed8 now it is outOfScreenTabsRenderingPages.

To reduce style computation I need to optimize CSS codes but I still don't know any available inspector to investigate bottleneck in CSS...

[DarthGandalf]

(back to ESR)

I've installed fc18b5c4fd92ba86e51d43c4073cd3c32e4dfb30 and tried to scroll through all my tabs from end to end with a stopwatch.

outOfScreenTabsRenderingPages=0 - 10s outOfScreenTabsRenderingPages=1 - 17s outOfScreenTabsRenderingPages=2 - 19s outOfScreenTabsRenderingPages=10 - 36s

in 3.9.22 - 5s

[piroor]

Thanks, I've found a bottleneck. Could you try with the commit 29e41d0 with outOfScreenTabsRenderingPages=9999999 ? Please note that outOfScreenTabsRenderingPages=10 or something will make things slow because it causes large number creation and deletion of DOM nodes. 9999999 means that building all DOM nodes at once and don't create/delete nodes on scrolling.

[DarthGandalf]

Now it's 7-8s with 9999999

[piroor]

OK, I've introduced a special mode with the commit 47a48de. Now outOfScreenTabsRenderingPages=-1 should work almost same as TST 3.x, about DOM nodes preparation.

[gtimbo]

@gtimbo It is intentionally shown to indicate there are more tabs after the visible area. However it should appear at the bottom of the view port and something wrong on your environment. Do you use any user style sheet to customize? Could you share exported config to more inspection?

Note that I've raised a new issue as my problem is unrelated to this one.

[DarthGandalf]

it's still 8s with outOfScreenTabsRenderingPages=-1, while 3.9.22 did it in 4-5s

Profile with 3.9.22: Firefox 2024-03-15 16.30 profile.json.gz

Profile with outOfScreenTabsRenderingPages=-1: Firefox 2024-03-15 16.35 profile.json.gz

In both cases I:

1. dragged the scrollbar to very bottom
2. started recording
3. clicked on the very top of scrollbar
4. held mouse clicked while it's scrolling to the top
5. stopped recording

[piroor]

More changes to reduce dynamic style computation are introduced with the commit 1eb34ec. I've confirmed that the number of style computation is quite reduced by this change. On my environment continuous scrolling looks to be accelerated, even with outOfScreenTabsRenderingPages=1.

[ssergiienko]

Is it possible to add opt-out switch for new virtual scrolling mode, at least till it mature enough. Because as of now, it looks like it gives zero benefits (or I misunderstand something?) and forces to stick with v3

[piroor]

I've introduced more changes to optimize outOfScreenTabsRenderingPages=-1 mode skipping some operations required on virtual scrolling. On my case rendering of Gecko itself is the last bottleneck so I think the mode is quite similar to the behavior on TST 3.x.

Because as of now, it looks like it gives zero benefits (or I misunderstand something?) and forces to stick with v3

Virtual scrolling is actually effective to accelerate frequent opening/closing the sidebar. When there are very large number tabs without virtual scrolling, old TST freeze for 10-20 seconds (or some minutes) when the sidebar is opened. It actually frustrated people including me on some situations: when they accidentally switch the sidebar panel (to Bookmarks, History, or others), when they restart the browser, when they reopen accidentally closed browser windows, and more. Moreover, to accelerate initialization process of the sidebar TST 3.x had the cache mechanism which contained full HTML source of the sidebar page ate the RAM and disk. Virtual scrolling has been introduced to solve such problems.

On the other hand, continuous scrolling like keeping mouse button pressed on the scrollbar is most major unfriendly case to virtual scrolling. On intermittent scrolling by mouse wheel or something, TST can render tabs while you are moving your finger from an edge to another so people may not sense the delay to render tabs, but continuous scrolling does not give time to TST to do such a trick. Scrolling by dragging the thumb of the scrollbar is also unfriendly case to virtual scrolling. Firefox dispatches "scroll" event when you stop your mouse, it means that TST cannot know which tabs need to be rendered until the scroll position is finally determined, thus sorry TST may expose a blank space while rendering for some seconds.

On my use case benefits of virtual scrolling is larger than drawbacks: waiting for long long seconds is painfully stressful for me, such a pain disallowed me to easy show/hide the sidebar to expand visible area for some webpages with wide contents.

[DarthGandalf]

I open the side bar ~never, except when accidentally closed it and on startup. But with my number of tabs I scroll up and down VERY often. So the tradeoff for me is definitely to make scrolling faster, even if startup is not as fast

I've introduced more changes to optimize outOfScreenTabsRenderingPages=-1

I'll be able to test it tomorrow

[ssergiienko]

@piroor Thank for explanation!

I've introduced more changes to optimize outOfScreenTabsRenderingPages=-1 mode skipping some operations required on virtual scrolling. On my case rendering of Gecko itself is the last bottleneck so I think the mode is quite similar to the behavior on TST 3.x.

I did tested just now TST from latest commit and can confirm that with all recent optimizations it's now very close to TST 3.x scrolling performance. So it's now become usable again) I also tried multiple values for outOfScreenTabsRenderingPages and for me

• with default 1 - very noticeable visual delay (is it a good default then?)
• starting from 7 and more - almost no visual delay
• with -1 , hard to say, it's either exactly like with 7 or a bit better, subjectively

TST 3.x had the cache mechanism which contained full HTML source of the sidebar page ate the RAM and disk

Do I understand correctly that

• Optimize tree restoration with cache and Persistent tree cache are no longer effective with TST 4.x ? Should they be removed then?
• RAM usage with 4.x is expected to be a bit lower?

Regarding use-case(hope it will be useful for you), for me priorities are exactly opposite. I almost never close/open TST so don't care very much about these timings as that happens once in a weeks when I restart FF. Yes, habit were initially driven by slowness of that, but not only, as frequent looking at tree helps to remind myself about context of current child tabs, so I never close TST. But scrolling from top to bottom happen hundred times a day because tree height is constantly growing during research activity and it's usually very fast with "hyper-fast" wheel scrolling mouse, so it would be painful to have it with delays

[DarthGandalf]

I've tried d8fbd01 on ESR, and can partially confirm @ssergiienko 's findings.

With -1 it's slightly slower than 3.x, but much more usable already than 4.0, it now takes 6 seconds to scroll end to end. It's much slower with 0, 1, 9999999 and other numbers.

I almost never close/open TST, but not because it's slow, but because I switch tabs all the time.

[piroor]

The default value of outOfScreenTabsRenderingPages (1) is quite arbitrary. Too small number will expose blank area frequently with quick scrolling, and too large number will increase RAM and CPU usage needlessly. On my use case mainly using wheel scrolling, 1 looked large enough.

I've exposed the option as an expert option under the "Appearance" section, so I hope people who are quite stressed with the default value will customize it.

And, the cache option is still used by the background page (the main process.) On-memory cache is for windows reopened by Ctrl-Shift-N or Ctrl-Shift-T, and persistent cache is for the browser startup or updating of TST. Without the cache you'll still wait long time until the first view appears.

[irvinm]

@piroor will that variable be part of Firefox Sync or be unique to that browser instance?

[amandamana]

I use a Tab Mouse Wheel TST addon and changing tabs via the wheel has gotten really slow. Like 1.5-2 seconds slow. I'm trying to figure out the cause. Setting outOfScreenTabsRenderingPages to -1 has no effect. Should I try downgrading to 3.x? Any other secret settings ideas?

Edit: downgraded to 3.9.22 and tab switching by mouse is back to an initial latency of ~0.25 sec

[piroor]

@amandamana Could you paste configs of the Tree Style Tab Mouse Wheel? And how many tabs do you open when you saw the slow scrolling?

[piroor]

@irvinm Both options are synchronized between devices. Should it not synchronized?

[amandamana]

@amandamana Could you paste configs of the Tree Style Tab Mouse Wheel? And how many tabs do you open when you saw the slow scrolling?

I have about 500 tabs in my tree and most are unloaded.

[irvinm]

@irvinm Both options are synchronized between devices. Should it not synchronized?

No ... synchronized is good. Thanks for verifying.

[DarthGandalf]

Disclaimer: I'm not using Firefox Sync, here are just some thoughts.

For me v4.0.8 works perfectly now in FF 122.0.1 with the default setting of 1. It might be slightly slower than in v3, but the delay is barely noticeable. But the startup is now much faster than in v3.

But in my ESR profile on the different machine, as shown above, I have to use the setting of -1, otherwise TST is not usable at all.

So, I wouldn't want Sync to force me use the same setting on both instances.

[piroor]

Hmm... options mainly about performance looks to need to be configured for each device. Some performance options are already device specific, so it looks reasonable to make these options also device specific.

[piroor]

@amandamana Thanks! I've confirmed slowing down on the situation. The performance profile says there are two major bottlenecks. I'm trying to investigate more.

[piroor]

There was a bottleneck around operations to export tab information to helper addons - it frequently generate objects on every wheel scroll event even if there is no change around tabs. I've introduced changes to accelerate with cached results if there is no change about the tab. After these changes wheel scrolling become faster for me.

@amandamana Could you try the development build? https://github.com/piroor/treestyletab?tab=readme-ov-file#development-builds You need to activate the cache mechanism manually: TST options => Development => All Configs => set cacheAPITreeItems checked.

It is disabled by default because there is a concern: caches need to be cleared when something change happens on the tab but I may forget to do that on some situations. As the result API response possibly become "broken" after some operations. I think it still need to be tested more, so the cache feature is still disabled by default for more while.

[amandamana]

@amandamana Could you try the development build? Unfortunately, I don't notice any improvement in CI/CD #2477: Commit 4f6c438 over the earlier 4.x release. I tried unchecking and re-checking 'Optimize tree restoration with cache'

[piroor]

@amandamana Could you collect performance profile while you scrolling the wheel on TST's sidebar panel? You need to use the browser toolbox https://hacks.mozilla.org/2022/03/performance-tool-in-firefox-devtools-reloaded/ and choose "Firefox" from the "Settings" dropdown list. After that please share the saved profile.

[amandamana]

Sorry to sidetrack this, but all my TST settings got reset when removing the temporary addon. I had a lot of User Style Sheet customizations that I'm especially trying to recover. Would extension settings like these have been stored in the sessionstore.jsonlz4?

[piroor]

Oh... you just had to restart Firefox, then temporarily installed version is unloaded and previously installed release version of TST would be back. Sadly "uninstall" operation clears all stored configs and there is no way to get back configs, except import a JSON file including exported all configs (you had to create it before clearing) manually...

Edit: I've added a note about this point. https://github.com/piroor/treestyletab/commit/3dd3fe3180892e5c90fbc84143e3bb8228b44e38

[amandamana]

I have incremental file system backups, but have so far been unable to restore the correct file.

[piroor]

Extension settings are saved under the "storage" folder, and stored into SQLite database files. It is hard to restore from file restoration, instead restoring whole your Firefox profile is recommended.

[github-actions[bot]]

This issue has been closed due to no response within 14 days after labeled as "maybe fixed", 7 days after last reopened, and 7 days after last commented.

[awused]

I've found that increasing outOfScreenTabsRenderingPages is just a bad idea - you still get large blank spaces and it makes performance much worse. Scrolling up performs much worse than scrolling down for whatever reason - perhaps prepending elements is causing the layout code to run for more elements? I've just turned off virtual scrolling for now.

[piroor]

Some changes to reduce needless animation effects (introduced for #3572) may affect to this issue.

[awused]

4.0.17 definitely performs much better across the board. There are no longer any long pauses and any pauses I've noticed are down to a frame or two, but it does still have the problem of flickering where blank sections appear and are quickly filled. This happens even at high outOfScreenTabsRenderingPages values like 10. Something seems off as I'd expect a value that high to effectively eliminate the blank sections during normal scrolling.