Increase result limit for the number of entries

[YodaEmbedding]

Increase the result limit for the number of entries or provide a configuration option for it.

Rationale: It's useful to page up/down through the results sometimes.

---

For now, I have been manually overriding it via:

diff --git a/lua/smart-open/matching/multithread/create.lua b/lua/smart-open/matching/multithread/create.lua
index 0d07b4a..0a2650e 100644
--- a/lua/smart-open/matching/multithread/create.lua
+++ b/lua/smart-open/matching/multithread/create.lua
@@ -12,7 +12,7 @@ local function create_matcher(matching_algorithm, context)
   local slot_count = 4 -- threadpool thread count
   local last_processed_index = 0
   local top_entries = {} -- treat it as ascending (lowest relevance last)
-  local top_entry_count = 20
+  local top_entry_count = 1024
   local waiting_threads = 0
   local history_data_cache = {}

diff --git a/lua/smart-open/matching/multithread/process.lua b/lua/smart-open/matching/multithread/process.lua
index 08d769d..f881c64 100644
--- a/lua/smart-open/matching/multithread/process.lua
+++ b/lua/smart-open/matching/multithread/process.lua
@@ -1,6 +1,6 @@
 -- prompt, cancel_token, options, last_processed_index
 local function process(prompt, cancel_token, encoded_options, encoded_entries)
-  local result_limit = 20
+  local result_limit = 1024

   local ok, result = pcall(function()
     local options = vim.mpack.decode(encoded_options)

Also relevant

```diff diff --git a/lua/telescope/_extensions/smart_open/finder/finder.lua b/lua/telescope/_extensions/smart_open/finder/finder.lua index e4975fa..d27d61b 100644 --- a/lua/telescope/_extensions/smart_open/finder/finder.lua +++ b/lua/telescope/_extensions/smart_open/finder/finder.lua @@ -65,9 +65,9 @@ return function(history, opts, context) __call = function(_, prompt, process_result, process_complete) results = {} - if prompt == "" and #db_results >= 50 then + if prompt == "" and #db_results >= 200 then for _, result in pairs(db_results) do - priority_insert(results, 50, result, function(x) + priority_insert(results, 200, result, function(x) return x.base_score end) end @@ -90,7 +90,7 @@ return function(history, opts, context) local to_insert = vim.tbl_extend("keep", { ordinal = entry.relevance, display = opts.display, prompt = prompt }, entry) - priority_insert(results, 50, to_insert, function(e) + priority_insert(results, 200, to_insert, function(e) return e.relevance or e.base_score end) ```

[danielfalk]

The trouble is that this can hurt performance, especially if we're returning more entries from each job.

It would probably be better to increase the value from 50 in lua/telescope/_extensions/smart_open/finder/finder.lua to whatever you want it to be.

Unfortunately, increasing this value is also bad for performance because inserting entries into telescope is somewhat slow. Honestly, I think 50 is even too much for most cases I can think of. If you're moving up and down through 50 entries, what's your use case? I'm far more interested in improving the search so that the thing you're looking for is in the top 10.

I'd be open to increasing the result_limit and the priority queue limit of 50 in finder.lua a little bit if necessary if anyone wants to run experiments on performance and can find an ideal value here.

[zcysxy]

Second this, as it currently leaves a huge blank in some Telescope layouts

[carlosflorencio]

Also looking into having this configurable.

[danielfalk]

I haven't prioritized this because it isn't clear what such a setting is useful for beyond filling out a larger results pane. If the entry you're looking for isn't in the top 20 items, then how efficient is it for anybody to scroll through looking for what you need? That's what I meant when I asked about a use case. I just think I'm missing something if the entry you need is not in the top 20 items. So there's a couple things that make me hesitate on this:

1. If we make the result set configurable, then will there be more performance complaints?
2. Maybe this points to a deeper issue and the results are so useless that people have to page through over 20 results to get to what they need! I'd rather get feedback that can help the results be more useful than publish a setting that encourages others to work around the main issue. I worry that would just lead to less useful feedback overall on the more important question of search result quality.

I may just be missing something, but that's why I want to hear more info.

[YodaEmbedding]

Scrolling is beneficial when the user doesn't know exactly what they're looking for.

Example: Consider a large codebase with 10000+ files. The user wants something related to ".../sdk/.../components/...". But there may be 100s of such components.

The fastest way to figure out which files may be relevant is some sort of fuzzy search + scroll through results. Another alternative:

fd | fzf
# Now fuzzy search for "sdk/components/"

...but then that adds a few extra steps of: (i) open another terminal, (ii) run command, (iii) once a list of candidate files is determined, open each candidate manually in the editor by fuzzy searching for it again.

This use case is similar to grep-style searches through a large codebase, though it's a much more lightweight and quicker alternative since the user already has some idea of the what part of the filename is.

[carlosflorencio]

My use case is very similar to @YodaEmbedding. I'm not trying to find anything particularly at the beginning of the search but I need to filter a set of files and navigate through them (previewing the contents) until I can further refine the search filter.

[danielfalk]

That helps, and it's a reasonable scenario. I'm not sure if this tool is the best for that because all you want in that case is a huge list of files more or less in the same directory. Relevance plays virtually no part there. Though I wonder...if you don't know what you're looking for except by name, how do you know when you've found it?

Still, I think part of the point for this plugin is to not have to have a bunch of different keybinds to jump between files. You shouldn't have to think "Is it a buffer? A recent file? ..." etc. You've brought up a useful scenario and has gotten several upvotes. Let me prioritize this, but I don't really think I can do much about the speed, and it may be slower. I'll just mention that in the docs.

[carlosflorencio]

Like you said, having the same key binding to find files is a big plus. In my case I just need a bit more than 20 results when I’m not sure what I’m searching for yet.

I’ve been using 50 results (using the code changes provided by @YodaEmbedding above) and didn’t noticed any slowdowns.

[danielfalk]

OK, this is out on the 0.2.x branch. The option is called result_limit

[carlosflorencio]

Thanks @danielfalk, works as expected :)