FR: add an option to customize *initial* sorting of the results, when the query is still empty

[chrisgrieser]

Is your feature request related to a problem? Please describe. Currently, the two options to change the sorting of files are file_sorter and tiebreak. However, as far as I can tell, both of those only take effect when the user as entered a search query (i.e., a non-empty TelescopePrompt). When the query is still empty, no sorting takes place, and the alphabetical sorting provided by fd is applied:

This initial order is often not desirable, as this results for example in Makefile and README always being placed at the top, even though they are files that are rarely accessed in comparison to other files.

Describe the solution you'd like Add a config option initial_sorting, which has the same signature as tiebreak. This function should sort results when the query is empty, and otherwise take no effect.

This way, you can write a short function that, for example, sorts lua files before markdown files.

Describe alternatives you've considered The only alternative I could think of is a really hacky workaround:

1. when calling find_files, put a space into the input field via autocmd.
2. this space has no effect on file_sorter (as far as I can tell), but it does trigger tiebreak.
3. Set tiebreak to my "sort lua files before md files" function.

[jamestrew]

I'm not sure about this. I have a few different thoughts but to put it concisely...

file_sorter does score entries on initial startup. It's probably just not affecting anything based on your file_sorter's scoring function. I suppose you can subclass you file_sorter to apply some custom scoring when the prompt is empty if you really wanted to. This is better than hijacking tiebreak in my opinion. Telescope currently only handles max 250 entries and tiebreak operates on this subset of results. So using tiebreak, you'll be fully sorting (I think) the first 250 results from fd (which is technically non-deterministic btw). So it could be bad performance wise and still give you terrible results.

If you're looking for ways to get relevant files to show up at the top of your results, I don't think sorting/scoring based on file names is that great anyways when more context aware approaches like telescope-frecency and https://github.com/danielfalk/smart-open.nvim exist.

[chrisgrieser]

To be clear, I wasn't advocating for the workaround with tiebreak that I described, just mentioned it as the only alternative I could up with. And precisely because it is so problematic, I made this feature request for a proper solution.

I tried the zf, fzy, and fzf native sorters already, and as far as I can remember, none of them affected the initial sorting for me.

smart_open does add too many other things which are not necessarily what you want. Telescope sorting works just fine for me, except for when the query is empty.

The performance impact issue I agree with. Nonetheless, not everyone is using telescope on big projects, in smaller projects the impact is negligible. By making any initial sorting function opt-in, you could decide for yourself whether the trade-off is worth it for you.

[jamestrew]

Yeah so just because those sorters actually affect the initial sorting, it doesn't mean they aren't scoring things initially. They're just giving everything the same score.

I'm not sure it makes sense to add an option for a separate initial sorter just because the initial scoring of those sorters aren't to your liking.

I think the best approach would be to subclass your preferred file_sorter and augment the scoring_function when the prompt is empty.

telescope-fzf-native for example exports a function to get the sorter. https://github.com/nvim-telescope/telescope-fzf-native.nvim/blob/6c921ca12321edaa773e324ef64ea301a1d0da62/lua/telescope/_extensions/fzf.lua#L144-L148

Then I think you can subclass it and do some custom scoring if the prompt is empty based on the line https://github.com/nvim-telescope/telescope-fzf-native.nvim/blob/6c921ca12321edaa773e324ef64ea301a1d0da62/lua/telescope/_extensions/fzf.lua#L95-L103

I mean there's a few other different ways of going about this.

• Writing a custom picker where you sort the results from fd first yourself
• using default_text=".lua$" or something similar to do some pre-filtering

I'm not sure if adding an initial_sorter type option into telescope makes sense or is worth the trouble.

[chrisgrieser]

using default_text=".lua$" or something similar to do some pre-filtering

Ah nice, didn't know about that one. Solves a few problems I had, thanks. Looks like it's undocumented though, as I cannot find it in the help docs?

---

Writing my own picker sounds like quite a lot of hoops to jump through just for some better initial sorting.

I'm not sure it makes sense to add an option for a separate initial sorter just because the initial scoring of those sorters aren't to your liking.

I somewhat get your point. Though when in practice none of the sorters implement initial sorting, it kinda feels like a diffusion-of-responsibility situation. I'll open FRs at the sorters though, maybe someone also sees the benefit of this.

[jamestrew]

Maybe I muddled the waters of responsibility a little bit.

The point I wanted to get across is not really that fzf-native or fzf-native should implement better initial sorting. Their whole thing is scoring results against a provided prompt. But, at the same time, what you're requesting for, sorting initially, is a mechanism that already exists - it's just not utilized by these sorters in any meaningful way. Probably due to how subjective an initial sort would be. Hence the idea of subclassing your preferred sorter to implement the initial scoring to your preference.

[chrisgrieser]

Tbh, I wouldn't know how to implement that. Is there an example somewhere where someone has done such a subclass so I can see how that is done?

[jamestrew]

I don't think so. It's a pretty niche request. Usually people just want frecency.

But I think something like this works

local init_sort_fd = function(picker_opts, sorter_opts)
  picker_opts = picker_opts or {}
  sorter_opts = sorter_opts or {}

  local fzf = require("telescope").extensions.fzf.native_fzf_sorter(sorter_opts)
  local my_fzf = {}
  setmetatable(my_fzf, { __index = fzf })

  ---@param prompt string
  ---@param line string
  ---@return number score number from 1 to 0. lower the number the better. -1 will filter out the entry though.
  function my_fzf:scoring_function(prompt, line)
    local score = fzf.scoring_function(self, prompt, line)
    if prompt == "" then
      -- custom scoring logic
      if line:find(".lua$") then
        score = 0.5
      end
    end
    return score
  end

  picker_opts = vim.tbl_deep_extend("force", picker_opts, {
    find_command = { "fd", "-tf" }, -- just for my testing purposes (fd seems more deterministic than rg)
    sorter = my_fzf,
  })

  require("telescope.builtin").find_files(picker_opts)
end

I used fzf-native here but I just checked with zf-native and it looks like it also exports the sorter so it'll be the exact same but change the name.

[kniteli]

I wrote a project file search picker where I wanted the most recently edited files at the top, barring any other filters. Basically, tiebreak on edit age, ascending (I utilized oldfiles and active buffers to populate initially), before you start typing.

This however is actually not possible due to a frankly confusing condition that tiebreaking is disabled for any entry with a score >= 1. Turns out this is every entry for which there is no overlap with the query (at least for fzf-native which I was using). relevant code, the expression score < 1 is the cause here

I don't really buy the argument that using tiebreak for initial sort is wrong. This means tiebreaking is disabled when it's at its most valuable: before the user has typed any search string, meaning every entry has the same score of 1. If you're not tiebreaking in that situation, when exactly are you supposed to use it?

I've got a fix over here: https://github.com/nvim-telescope/telescope.nvim/compare/master...kniteli:telescope.nvim:fix-tiebreak-with-no-input

Curious if there's any insight into why that score check is necessary? I've been running with this fix applied for a bit now and haven't noticed any oddities. Doesn't seem like it does anything since the default tiebreak is by ordinal anyway.

This also isn't solvable with plugins, except by writing your own tiebreak into your own custom sorter.

[jamestrew]

So using tiebreak, you'll be fully sorting (I think) the first 250 results from fd (which is technically non-deterministic btw). So it could be bad performance wise and still give you terrible results.

I think I was actually wrong about this. Either way, we can't really make that change to score < 1 since that would be a pretty huge breaking change. For anyone currently using tiebreak, removing this would mean going from occasionally tiebreaking to basically always tiebreaking at the start depending on their choice of sorter. This could completely destroy performance for larger searches.

If you still want to have an initial sort order, you can write a sorter that does this. Or just sorts the initial list before passing it to result. The latter, we do in several places throughout telescope core and extensions. I probably should've recommended this first.

[chrisgrieser]

In case anyone finds this via search, I found a much better solution for this.

While fd does not allow for any kind of result sorting (it has no --sort flag), in fact, rg does have such a flag, and also the option --files which makes rg spit out only files it would search, without actually searching them. As rg has the same behavior regarding ignore files, rg can actually be used as a (bit less performant) drop-in replacement for fd to achieve initial sorting by mdate:

find_files = {
    find_command = { "rg", "--no-config", "--files", "--sortr=modified" },
},

[atimofeev]

While setting from previous comment works great for initial results, how do I sort by modified results of get_fuzzy_file? Is this even possible?