open new tab (`t`) dialog very slow to load suggestions

[blendmaster]

Error details

SurfingKeys: Version 0.9.66 from AMO

Browser: Firefox 78.0.2 (64-bit) Linux.

URL: All urls

Context

On any page, pressing t with the default bindings brings up the in-page location bar near instantly. However, typing any letter results in a few seconds long pause and CPU usage spike while before it loads any suggestions, making the t functionality in SurfingKeys near unusable compared to the native location bar. Even once the initial suggestions load, typing any additional letters or hitting tab makes the browser unresponsive for a few seconds.

This is on my default firefox profile with plenty of other extensions and a large amount of history/bookmarks. I haven't tried bisecting my extensions or settings yet to see if I can eliminate the slowdown. However, the similar functionality of vimium's "omnibar" worked fine, as does the native firefox bar, so I suspect something with surfingkey's history loading specifically makes the process slow/expensive.

[ALPHA-60]

Similar problem here, but happens immediately after pressing t - the initial list of suggestions takes around 3 seconds to load, and the CPU usage spikes. After that, I can type normally. I unconsciously switched to using built-in Ctrl-t.

Browser: Firefox 82.0.2 (64-bit) FreeBSD Surfingeys: 0.9.68

[euglevi]

Surfingkeys: 0.9.69 Firefox: 86.0 (64-bit) on Ubuntu 20.04

I have the same problem. With t or go, suggestions take a long time (4-5s) to load. Then, the bar freezes for 1s for the suggestions to show up before turning back to normal again. I do not find the same issue while using search aliases or bookmarks. However, I do find the same when searching trough history, so this has clearly something to do with how Surfingkeys handles this process. This did not happen with Vimium, Tridactyl or other similar extension.

By the way, the extension is great! Thank you for your effort.

[paniash]

I have the same issue as well. Is there a way to bind the default firefox binding <Ctrl-t> to t? @brookhong

It works fine on chromium and brave. This issue is specific to firefox AFAIK.

Browser: Firefox 88.0.1 (64-bit) on Artix Linux Surfingkeys: 0.9.74

[perfectayush]

I have the same issue in Firefox 89 (and previous versions too). I almost exclusively use Ctrl + L for this reason.

Today, I spent some time trying to understand what's going on using some performance tools in about:debugging. Looks like a lot of gc happens when the omnibar is open in firefox, it also reduces framerate from ~ 60fps to ~ 0.6 fps.

Steps to do:

1. Open about:debugging. Click on This firefox and Scroll/Find SurfingKeys. Click Inspect
2. This open a web inspector like console. Go to performance.
3. Performance tab has a sidebar, on top of which there is a toggle recording state for performance recording button. The button look like a stopwatch. Click on that.
4. Switch to a open tab. Open omnibar, type something. Observe the slowness. Switch back to Toolbox tab for extension.
5. Toggle the recording state again with the stopwatch button. You will have the profile report as above.

[perfectayush]

Okay, I built up on the above performance analysis, this took a while.

Due to multiprocessing in Firefox, the extension loads background.js in another process and some other parts like omnibar.js in the tab process. This also means that performance recording of both need to be analyzed in 2 different inspectors.

After doing that analysis properly, I realized that having a very big history causes slowdown issues with SurfingKeys omnibar in Firefox. The slowdown is mainly in 2 places:

1. function _getHistory in background.js: This function calls a sort when sortByMostUsed is True. The sort in firefox which is implemented as Promise. When number of history items are large, around 85k in my case, there is a significant slowdown in this function. I tested by limiting history items to 2000 in the function using maxResults argument of chrome.history.search.
2. function _filterByTitleOrUrl in omnibar.js. This function calls filter over all the urls using regex generated by input string query in omnibar. Again this filter is significantly slowed when the url item are quite large.

To fix this issue, we will require to come up with strategy to speed up this sort and filter in these 2 functions. A couple of quick hacks, I could think of are:

1. To limit the items returned by _getHistory in some fashion, this will solve issue for filter in omnibar.js. Making this limit configurable. This however can lead to weird results as complete history won't be loaded.
2. Deleting browsing history is another not so great option. Truncating/Archiving it in some fashion is another similar option.

[jahkeup]

To limit the items returned by _getHistory in some fashion, this will solve issue for filter in omnibar.js. Making this limit configurable. This however can lead to weird results as complete history won't be loaded.

I wonder if icremental loads could happen after exhausting the current list of results - maybe either asynchronously after reaching some threshold (only if results come already sorted by MRU) or by spawning another query after the user pauses. Upon receiving the results from either kind of query, the results could be merged in the list of offered results or replace the existing items entirely (with care to not disrupt a user's selection).

There's probably also a hybrid approach as well where a very small list (again, MRU) is initially searched while another background query resolves "the rest" of the results from deeper history.

I'm not familiar with the code and prior issues made, so forgive me if this is asked/answered/impossible: is it possible to cache the history items, and periodically refresh, centrally? And incrementally fill in the deeper wells of history over time?

[nineKnight]

Hi @brookhong , a loyal fan here, after vimium. Any further move to fix this?

[shihabdider]

I had a similar issue in chrome where loading the history would take so long that it appeared as if it wasn't working at all (because there is no loading indicator for updating the search -- another thing that would be a nice feature). Simply disabling and enabling the extension seemed to have fixed it.