Optimize opening files in folders with large filecounts

[emeraldtip]

I have a little bit of a problem. My Downloads folder is over 400 gigabytes. Over 440 000 files and folders (including subfolders) in the folder. And it's on a harddrive. Yes, I do know that that is very bad, not good in the slightest and I should deal with that, but that takes time.

Now the issue arises when I try to edit a single file in the Downloads folder, for example some simple script I got from the internet. When I open said singular file the whole editor hangs for about a minute as it's trying to load in all the folders and files from the Downloads directory.

Would it be possible to have the editor not load all the files in, when opening a single file alone and then have an option to click - load the location of the current file on the sidebar? Or have the folder loading happen asynchronously so that you can already get to editing your file, while the sidebar shows a loading symbol? Or something else?

[SpartanJ]

Hi! I'll have to try to set up something similar to understand where it is hanging. Directory tree is done asynchronously (otherwise this would happen basically any time you load a big folder tree), and when opening a single file the directory tree is not being scanned (and the option to load current file dir and scan it actually already exists in Settings -› Tools). So my guess is that there is some file system call that usually would be very cheap but in your case is extremely slow and it's happening in the rendering tree or there's a lock that stops the rendering thread to continue processing. So the folder that contains the file also contains hundreds of thousands of files (I mean that directory, not the full sub tree)? I think I don't even want to test it, it hurts my poor file system even for thinking something like this... also... you're a monster for having that folder 😂.

[SpartanJ]

BTW: If you happen to know how to debug C++ you could help me by attaching a debugger to the process and interrupting while this hangs, the stack trace would be more than enough information to know what and where to fix the issue.

[SpartanJ]

Oh, I know what it is. The problem is that the file tree wants to display the directory contents where the file resides and it will actually fetch the root directory contents in the main thread to display the current selected file. I'll have to refactor quite a bit of code to make this work, but I'll do it.

[emeraldtip]

Aight, thx

[SpartanJ]

I pushed a fix for your particular issue, I cannot test it exactly for your particular case but it should work based on my own tests. Next nightly build since this post will have the fix. Hopefully nothing weird introduces these changes (I had to do a good amount of refactor and performance improvements).