Closed gegoune closed 2 years ago
It should open quickly, there is no perceivable delay for me. There are no redraws due to the container, but there will be redraws after the initial display for git status and diagnostic symbols.
If you are talking about a netrw hijack then nvim will first load the empty directory buffer and then it will quickly be replaced with neo-tree. It is a perceivable flashing of windows if you are looking closely.
This sort of issue is often the result of interactions with some other plugins, like anything that affects working directories or resizes windows automatically.
I will investigate further in next few days then. Wasn't talking about netrw hijack, just simple :Neotree reveal
(which I now replaced with :Neotree
in my mapping).
Oddly running :Neotree
from cmd mode seems to be faster, almost instantaneousness, while mapping such as
vim.keymap.set('n', '<leader>vf', ':Neotree<CR>', { silent = true, desc = 'Neotree: reveal' })
clearly blinks the cursor and has a delay.
Edit: just tried on minimal config - no delay and blinking. Will bisect my config and report back.
I have the same issue. the delay is a couple seconds for me on a large project. I suspect the renderer for new container
or fade
feature has a performance impact.
If you want to see if it is the container, please update to the latest release (just pushed) and try these steps one at a time:
Add resize_timer_interval = 5000
(5 seconds) to your config:
require('neo-tree').setup({
resize_timer_interval = 5000,
})
If that has no effect, change your renderers
to the old config without containers:
require('neo-tree').setup({
resize_timer_interval = 5000,
renderers = {
directory = {
{ "indent" },
{ "icon" },
{ "current_filter" },
{ "name" },
-- {
-- "symlink_target",
-- highlight = "NeoTreeSymbolicLinkTarget",
-- },
{ "clipboard" },
{ "diagnostics", errors_only = true },
--{ "git_status" },
},
file = {
{ "indent" },
{ "icon" },
{
"name",
use_git_status_colors = true,
},
-- {
-- "symlink_target",
-- highlight = "NeoTreeSymbolicLinkTarget",
-- },
{ "bufnr" },
{ "clipboard" },
{ "diagnostics" },
{ "git_status" },
},
},
})
If that also has no effect, it's probably something else.
I reverted to the old config too and tried the resize time interval, still have the delay. so probably something else. my config:
require('neo-tree').setup {
close_if_last_window = true,
popup_border_style = 'rounded',
use_popups_for_input = true,
enable_git_status = true,
enable_diagnostics = false,
resize_timer_interval = 5000,
default_component_configs = {
indent = {
indent_size = 2,
padding = 0,
with_markers = true,
indent_marker = '│',
last_indent_marker = '└',
highlight = 'NeoTreeIndentMarker',
with_expanders = true,
expander_collapsed = '',
expander_expanded = '',
expander_highlight = 'NeoTreeExpander',
},
icon = {
folder_closed = '',
folder_open = '',
folder_empty = '',
default = '',
},
name = {
trailing_slash = false,
use_git_status_colors = true,
},
git_status = {
symbols = {
-- Change type
added = '+',
deleted = '',
modified = '',
renamed = '➜',
-- Status type
untracked = '',
ignored = '',
unstaged = '',
staged = '✓',
conflict = '',
},
},
},
renderers = {
directory = {
{ 'indent' },
{ 'icon' },
{ 'current_filter' },
{
'name',
use_git_status_colors = true,
},
{
'symlink_target',
highlight = 'NeoTreeSymbolicLinkTarget',
},
{ 'clipboard' },
{ 'git_status' },
},
file = {
{ 'indent' },
{ 'icon' },
{
'name',
use_git_status_colors = true,
},
{
'symlink_target',
highlight = 'NeoTreeSymbolicLinkTarget',
},
{ 'bufnr' },
{ 'clipboard' },
{ 'git_status' },
},
},
window = {
position = 'right',
width = 40,
mappings = {
['o'] = 'open',
['<cr>'] = 'open',
['S'] = 'open_split',
['s'] = 'open_vsplit',
['R'] = 'refresh',
['a'] = 'add',
['A'] = 'add_directory',
['d'] = 'delete',
['r'] = 'rename',
['c'] = 'copy_to_clipboard',
['x'] = 'cut_to_clipboard',
['p'] = 'paste_from_clipboard',
['q'] = 'close_window',
['y'] = function(state)
local path = state.tree:get_node():get_id()
print('Copied to clipboard: ' .. vim.fn.fnameescape(vim.fn.fnamemodify(path, ':~:.')))
vim.fn.system('pbcopy', vim.fn.fnameescape(vim.fn.fnamemodify(path, ':~:.')))
end,
['Y'] = function(state)
local path = state.tree:get_node():get_id()
print('Copied to clipboard: ' .. vim.fn.fnameescape(path))
vim.fn.system('pbcopy', vim.fn.fnameescape(path))
end,
['<bs>'] = 'none',
['.'] = 'none',
['m'] = 'none',
},
},
nesting_rules = {},
filesystem = {
window = {
mappings = {
['H'] = 'toggle_hidden',
['I'] = 'toggle_gitignore',
['C'] = 'close_node',
['z'] = 'close_all_nodes',
['<C-x>'] = 'clear_filter',
['f'] = 'filter_on_submit',
['/'] = 'fuzzy_finder',
['h'] = 'navigate_up',
['l'] = 'set_root',
},
},
filtered_items = {
visible = false,
hide_dotfiles = false,
hide_gitignored = false,
hide_by_name = {
'.DS_Store',
'thumbs.db',
'.git',
},
never_show = {
'.DS_Store',
'thumbs.db',
'.git',
},
},
follow_current_file = false,
hijack_netrw_behavior = 'open_default',
use_libuv_file_watcher = true,
},
buffers = {
bind_to_cwd = false,
show_unloaded = true,
},
git_status = {
window = {
mappings = {
['A'] = 'git_add_all',
['u'] = 'git_unstage_file',
['a'] = 'git_add_file',
['d'] = 'git_revert_file',
['gc'] = 'git_commit',
['gp'] = 'git_push',
},
},
},
}
vim.cmd [[
let g:neo_tree_remove_legacy_commands = 1
nnoremap <silent> <C-p> <cmd>Neotree toggle reveal<cr>
nnoremap <silent> <C-b> <cmd>Neotree toggle reveal float buffers<cr>
nnoremap <silent> <C-g> <cmd>Neotree toggle reveal float git_status<cr>
]]
@sandangel Can I ask how many files are in that directory? Can you try opening the tree in other directories to see if it has the same issue? Is it the same delay if you close and re-open it a second time?
Another thing that would be helpful is to turn on logging and copy the logs here from the time frame where Neotree was opening: https://github.com/nvim-neo-tree/neo-tree.nvim/wiki/Troubleshooting
In the case of a delay that is several seconds, that should tell me what the issue is. It's not as good for millisecond delays, but it can still offer hints if I see things like repeated events that is causing excessive refreshes.
My config (difference with what's on the README):
use_libuv_file_watcher = true, -- This will use the OS level file watchers to detect changes
I tried it with a large codebase https://github.com/kubernetes/kubernetes and it takes some time to find a file, the logs that I captured for Neotree reveal
are https://pastebin.com/raw/GPEbfc7w
I tried to profile it with http://lua-users.org/wiki/PepperfishProfiler and I got this https://pastebin.com/raw/4HnPqWhM (same function with this setup)
local pepperfish_profiler = require('mauricio/pepperfish_profiler')
profiler = pepperfish_profiler.newProfiler('call', 10000)
function _G.mauricio_open()
profiler:start()
neotree.reveal_current_file("filesystem", false, "<bang>" == "!")
-- os.execute("sleep " .. tonumber(3))
profiler:stop()
local outfile = io.open("profile.txt", "w+")
profiler:report(outfile)
outfile:close()
print("profile complete!")
end
vim.api.nvim_set_keymap('n', '[ui]c', [[<Cmd>lua mauricio_open()<CR>]], { noremap = true})
The output doesn't show anything that could be taking a long time however I noticed that after the profiler finishes there's some brief period of time where I can move the cursor before the sidebar is opened, is the renderer async or scheduled?
The output doesn't show anything that could be taking a long time however I noticed that after the profiler finishes there's some brief period of time where I can move the cursor before the sidebar is opened, is the renderer async or scheduled?
Yes, that command just starts the async filesystem scanner, and then the rendering of those results is also debounced and scheduled. This makes it hard to profile (and to test).
Also, that api is deprecated. Use this instead: https://github.com/nvim-neo-tree/neo-tree.nvim/blob/32c20173cecf311412132512dd4915c07e3ab38f/lua/neo-tree/command/init.lua#L39
I looked at the log and I noticed a lot of fs_events that seem unusual. I think the file watcher misbehaves on some systems which is why it's not on by default. Can you try setting use_libuv_file_watcher = false
and telling me if it works better?
yeah it works better with the file watcher set to false, I think it's good enough for me, personally I close the navigator when I'm editing a file so I can trigger a refresh right before opening it again, thanks for checking
yeah it works better with the file watcher set to false, I think it's good enough for me, personally I close the navigator when I'm editing a file so I can trigger a refresh right before opening it again, thanks for checking
It actually refreshes every time you open it, so no need for a manual refresh if that's how you use it.
Can I ask how many files are in that directory?
230 directories, 968 files
I tried with libuv watcher = false but still have the same delay. The delay is shorter when open the second time. It also has some delay when I open the subdirectory
.
Wow @sandangel, that is an extreme delay! When you say it has "230 directories, 968 files", do you mean as direct descendants of the folder you are viewing, or the total files for the project, recursively?
Also, what OS are you on?
One last thing, can you please turn on logging, repeat what you did in the video, and copy or link to the logs here? https://github.com/nvim-neo-tree/neo-tree.nvim/wiki/Troubleshooting
that is the total files for the project recursively. I got the number by using tree
cli.
I'm running on MacOS 12.3, macbook pro 16" 2019.
This is the logs:
I have this issue on relatively small repository, my dot files:
$ git ls-files | wc -l
124
As mentioned in original post, there is no delay with nvim-tree. I have also tried with use_libuv_file_watcher = false
(had it true
originally) and didn't notice much (if any at all) difference.
Thanks @sandangel, those logs are very useful. It looks like the delay is actually in the listing (or maybe processing) of the files. It has nothing to do with rendering on the screen or git.
@gegoune are you also using MacOS?
@cseickel Yes, I am.
Another oddity that might be related. I can open either git_status
or buffer
immediately without any visual glitches (apart from blinking cursor in the corner) and switch between them while neo-tree window is open. But as soon as I switch to filesystem
tree view it seems like something gets drawn first and then gets quickly replaced by actual tree causing flash like glitch. I tried to catch it on screen cast but it might not be clearly visible.
Can everyone please update to the latest in main and add this to your config:
require('neo-tree').setup({
filesystem = {
async_directory_scan = false
},
})
...and tell me how it works?
I tried it and stopped the cursor flickering, I can't tell if the performance improved because it renders pretty fast in my workstation with both this flag set to true/false, however after reading the code in the async version I believe that repeated calls in https://github.com/nvim-neo-tree/neo-tree.nvim/commit/68be06876d88cbd13b1e42359c31e7fa15a1f3e3# to plenary's scan.scan_dir_async
with depth = 1 is not actually making it async, the flow that I understood is:
async_scan(parent) ->
scan.scan_dir_async(parent) ->
wait for on_exit callback ->
on_exit called, call async_scan() with the next item in `context.paths_to_load`
I'm not sure if I read your code correctly but the above is making the sync scanning process still sync with additional callback overhead, probably it'd be better to do:
local function async_scan(context, callback /* new */)
for every path in context.paths_to_load
async_scan(path) ->
scan.scan_dir_async(parent) ->
wait for on_exit callback -> increase counter and call `callback` when all the paths in context_paths are done
I also noticed that plenary's scan.scan_dir_async
does some sync work at the start of every call that's the same for all the calls, if searching is a problem you could also do what telescope does and delegate the work to a third party tool if available like rg.
@cseickel Flicker seems to be gone now, thanks! I will look into timing both neo-tree and nvim-tree, but nvim-tree seems bit snappier, almost immediate while neo-tree takes a very short while (for the same repo).
I reproduced the problem consistently by cloning a big repo and opening as many directories as I could, I tried to improve the async scan in https://github.com/nvim-neo-tree/neo-tree.nvim/pull/240 and even though the scanning finishes fast for 50+ opened directories there is something slow running before and after the scan.
For the before part I'm not sure if it's this call https://github.com/nvim-neo-tree/neo-tree.nvim/blob/68be06876d88cbd13b1e42359c31e7fa15a1f3e3/lua/neo-tree/sources/filesystem/lib/fs_scan.lua#L173
For the after part it could be https://github.com/nvim-neo-tree/neo-tree.nvim/blob/68be06876d88cbd13b1e42359c31e7fa15a1f3e3/lua/neo-tree/ui/renderer.lua#L774-L798, I saw that it lags in the logs:
@mauriciopoppe
I tried it and stopped the cursor flickering, I can't tell if the performance improved because it renders pretty fast in my workstation with both this flag set to true/false, however after reading the code in the async version I believe that repeated calls in 68be068 to plenary's
scan.scan_dir_async
with depth = 1 is not actually making it async, the flow that I understood is:async_scan(parent) -> scan.scan_dir_async(parent) -> wait for on_exit callback -> on_exit called, call async_scan() with the next item in `context.paths_to_load`
on_exit
is an asynchronous callback. Unless plenary is lying, it's definitely async. What it is not is parallel, and that's by design because the dirs have to be scanned in order, parents first. The goal here was not to speed up the overall process because it's not usually slow when scanning a local hd. The goal is just to not block nvim's UI if you do scan something that is unusually large or slow.
Your suggested code would make it parallel, but like I said I don't think it would be accurate that way.
At the end of the day, it was completely unnecessary for this to be async to begin with, at least in any situation I have tested, and it just adds a lot of overhead for no reason. If the sync version is not a problem for most people, I may make that the new default.
I also noticed that plenary's
scan.scan_dir_async
does some sync work at the start of every call that's the same for all the calls, if searching is a problem you could also do what telescope does and delegate the work to a third party tool if available like rg.
Plenary's search is horrible and I don't actually use it. If you look closely, you'll see that the code branches before it gets to that point when there is a filter applied. It goes to filter_external.lua
.
@mauriciopoppe
I reproduced the problem consistently by cloning a big repo and opening as many directories as I could, I tried to improve the async scan in #240 and even though the scanning finishes fast for 50+ opened directories there is something slow running before and after the scan.
Thanks so much for getting hands on with this, that is unbelievably helpful! I'll look at the code paths you identified and see what I can find.
Can you also tell me what OS you are on, and what nvim version?
You know @mauriciopoppe, your logs show a very unusual amount of before_render/after_render with any apparent cause that I can see. A filesystem refresh or navigate should have been logged but I don't see it. I think it's possible that there's an endless loop in a render that you are getting caught in, and it's only obvious when you try to open the tree.
Can you also tell me what OS you are on, and what nvim version?
I have an arm MacbookPro with macOS 12 and:
NVIM v0.6.1
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by brew@HMBRW-A-001-M1-004.local
Features: +acl +iconv +tui
See ":help feature-compile"
system vimrc file: "$VIM/sysinit.vim"
fall-back for $VIM: "/opt/homebrew/Cellar/neovim/0.6.1/share/nvim"
your logs show a very unusual amount of before_render/after_render with any apparent cause that I can see. A filesystem refresh or navigate should have been logged but I don't see it. I think it's possible that there's an endless loop in a render that you are getting caught in, and it's only obvious when you try to open the tree.
I'm not sure if it's because I opened tons of folders and the hooks are called for every one of them, I saw that the neo-tree window also lags when I attempt to open a folder with many others already opened
I'm not sure if it's because I opened tons of folders and the hooks are called for every one of them, I saw that the neo-tree window also lags when I attempt to open a folder with many others already opened
That's probably it, I see now that there is no log entry for a lazy loaded folder other than those before/after render events.
I have an arm MacbookPro with macOS 12
Thanks. So far it seems like this issue only affects Mac and Windows users. Of course, this sample size is not at all statistically significant...
So far it seems like this issue only affects Mac and Windows users.
At work I have a Linux workstation and I see the same problem.
Please dismiss these logs https://github.com/nvim-neo-tree/neo-tree.nvim/issues/236#issuecomment-1088270985, I think I copied parts of the log that I used for debugging and aren't related with this issue
I have more logs with nanosecond precision after applying this patch to the log
local _, ns = vim.loop.gettimeofday()
-- Output to log file
if config.use_file then
local str = string.format(
"[%-6s%s:%06d] %s: %s\n",
nameupper,
os.date("%H:%M:%S"),
ns,
lineinfo,
msg
)
-- ...
A lot of time was spent rendering the UI, after reading the render code it's because it's always rendered from scratch https://github.com/MunifTanjim/nui.nvim/blob/main/lua/nui/tree/init.lua#L360, it looks like some of the performance improvements can be done in that repo instead
I forgot to say that my setup is like this:
log.trace("[mauricio] nuidraw start")
state.tree:render()
log.trace("[mauricio] nuidraw end")
I saw that you override the prepare_node
function so some profiling might be needed there too
A lot of time was spent rendering the UI, after reading the render code it's because it's always rendered from scratch https://github.com/MunifTanjim/nui.nvim/blob/main/lua/nui/tree/init.lua#L360, it looks like some of the performance improvements can be done in that repo instead
That is incredibly helpful. I am really surprised that this step takes so long. Can you copy and paste the contents of the Neo-tree buffer that is being rendered in those logs?
I just released a bunch of fixes that may resolve the issues. Please let me know how it works for you @mauriciopoppe @sandangel @gegoune
The tree:render()
delays may be the same cause as the memory leak, which is that we didn't think to turn off undolevels
for the neo-tree buffer. I'm just guessing, but it's worth checking out.
@cseickel it's still the same for me. now it even doesn't show up and makes my neovim instance freeze sometimes. When it shows up, if I open fzf-lua window, it even makes the fzf-lua window very unresponsive
One thing I noticed quickly is that this issue seems to be gone now! I also think that first invocation of filesystem
is still bit slower than ivim-tree's but consequent ones are pretty instantaneous. Will test some more, but thanks for that already!
@sandangel
@cseickel it's still the same for me. now it even doesn't show up and makes my neovim instance freeze sometimes.
It sounds like I was too optimistic and I should make the sync/async choice configurable.
When it shows up, if I open fzf-lua window, it even makes the fzf-lua window very unresponsive
That is really unusual. Can you look at top
or a task manager while this is happening and tell me if the cpu or memory is going crazy when this happens?
Can you share a minimal config to reproduce this? If you clone this repo and open a tree there does it happen, or does it have to be something huge to see a delay?
One thing I noticed quickly is that this issue seems to be gone now! I also think that first invocation of
filesystem
is still bit slower than ivim-tree's but consequent ones are pretty instantaneous. Will test some more, but thanks for that already!
What do you mean by "a bit slower", are we talking a few milliseconds or seconds? From your last video it looked pretty fast, so I'd call it done if it got any faster than that.
@sandangel if you update to the latest in main, you can now add this option to go back to the old behavior of always running this async:
require('neo-tree').setup({
filesystem = {
async_directory_scan = "always"
},
})
What do you mean by "a bit slower", are we talking a few milliseconds or seconds? From your last video it looked pretty fast, so I'd call it done if it got any faster than that.
It's pretty hard to tell, but nvim-tree feels like it opens straight away, while for neo-tree there is a slight but noticeable delay, I would say around 500ms, but it's very hard to tell. Can you think of a way to measure it somehow? On the other hand, I think it's plenty fast and am not sure if it's worth pursuing it any further. Also, I think that delay is only happening on very first invocation, all following ones are as nvim-tree fast. Am I making it up or is there a state or something held after initial open? Was testing now with `async_directory_scan = "auto".
Also, I think that delay is only happening on very first invocation, all following ones are as nvim-tree fast. Am I making it up or is there a state or something held after initial open?
Yes, the state is held when you close the window. It will show you that state on the next open and do a background refresh if needed. If nothing changed since the last open, you wouldn't even know that refresh happened.
Download video: https://ufile.io/rmx8wjom
Sorry it's a long video so I can not upload here but use https://ufile.io/ Hope it is not a malicious link
This is the trace log
Hi, apologies for poor issue here but don't have much time at the moment so putting it here in case it will easily indicate what the problem is. Will try to update this issue with more meaningful information next week.
When opening filesystem tree there is a very small but noticeable delay (before window opens) during which my cursor blinks few times (I don't have blinking cursor in my terminal). Delay is of about half a sec or even less, hard to tell and am not sure how to measure it. For reference both git_status and buffers trees open immediately without blinking the cursor (or at least it happens so fast that I can't see it). I have tested nvim-tree, which opens immediately for the same file in the same repository so I doubt it's IO issue. Perhaps some redrawing for new conteiners? I am not sure if it was like that before though, can try to bisect next week.