Enki scans many files when I open it.

[vi]

/# enki /etc/passwd

Enki spawns process that scans the entire system. My queer system may have circular mounts, hung filesystem locations (trying to access them may trigger uninterruptible sleep), automount locations and so on and I'd like to avoid scanning. Also it thrashes HDD and I want silence sometimes (I already know about SSD, but do not have it yet).

I sometimes use Enki just as a text editor (not as a little IDE) and start ot as root to edit few configs quickly. This scanning behaviour is not OK for this.

I think by default the scanning should kick in on the first usage of FuzzyOpener. Or this opportunistic background scanning should be able to see that it's going to scan something big and stop after, for example, 1000 syscalls, like I want to avoid user waiting, so I'll scan his project preemptively... Oops, it's a bit big project - can't scan it in 2 seconds. Let's stop and resume scanning only when user asks.. It may also need to handle directly_not_to_be_looked_into cases.

[vi]

By the way it uses a lot of stat syscalls (for each file). Is just readdir not enough? This way it should be more proportional to the number of directories scanned, not to the number of files.

[andreikop]

I use standard, well tested and cross-platform os.walk to scan files. It might be possible to scan the FS quicker but I think current speed is good enough.

I'd like to scan directories when project is opened, not when fuzzyopen is requested, because on big projects scan can be slow. (Even 0.5s is slow when opening the dialogue)

It is an open question when scan the directory and how to open project conveniently. If you are editing some file in /etc/ with root access, I agree that you don't want to "open project" and to scan the directory.

[vi]

cross-platform os.walk to scan files

Maybe https://github.com/benhoyt/scandir ?

I'd like to scan directories when project is opened, ...

Maybe as a minimum there should be command line options to avoid or force the scanning? In addition, when started as root or when current directory is $HOME or / it may also avoid scanning by default.

In general, it should not cease to be a simple lightweight text editor in favour of becoming IDE.

[andreikop]

I'm thinking about explicit project open action.

• No project - editor mode. No any scanning, no FuzzyOpen
• Project is explicitly opened - IDE mode. Directory is scanned. Enki remembers session (list of opened files, etc)

[vi]

Looks sane. One should be able to create or open such project/session from command line, or go from editor mode to IDE mode quickly (specifying the directory). When I specify -s (named session), I usually mean a project.

There maybe default heuristic mode that triggers --editor-mode or --ide-mode based on various factors (root or regular use, how current directory is named, typical file extensions, is there Makefile, configure.ac, build.xml, .gitignore in current (or parent) directory, was it opened before).

Example of heuristics pseudocode:

def is_project:
    projectness = 0
    if user is root: projectness -= 10
    if current_directory is / $HOME /tmp/* /proc/* /sys/* */.*/ (hidden) : projectness -= 10
    if current_directory has file configure.ac pom.xml project.clj build.xml CMakeLists.txt and so on and on and on: projectness += 20 (maybe varied depending on which file)
          (https://en.wikipedia.org/wiki/List_of_build_automation_software)
    if current_directory has .gitignore (.hgignore): projectness += 5
    if projectness > 15: return True
    if projectness <= -10: return False
    readdir current directory
    less than 20 files:
        readdir some subdirectories, especially "src" if exists
    if there are *.java files: projectness += 1
    if there are *.c files: projectness += 1
    ...
    if there are Makefile: projectness += 2
    ...
    if there are many other files without extension: projectness -= 2
    if there are symlinks: projectness -= 1
       (but don't turn readdir into readdir+stat solely to check for symlinks)
    if "src" directory exists: projectness += 3
    if "usr", "opt", "bin", "etc" subdirectories exist: projectness -= 6
    if there is "build" or "output" or "target" directory:
         projectness += 2
    else:
         if there are execulables or libraries: projectness -= 2
    return projectness > 0

[bjones1]

To me, @vi is looking for a "disable auto-scan" config option in the settings: he has certain cases where he doesn't want the filesystem scanned. For most users, I think the fuzzy open capability would be very helpful. What about setting a limit to the number of files the fuzzy opener will scan? That way, navigating to C:\ or / won't work through the entire hard drive, assuming a default limit of 100,000 files. Setting the limit to 0 would then disable it. There might be a note in the Locator saying that not all files were scanned if the limit is reached.

[vi]

A config options seems OK.

Maybe while scanning, some status may be shown, so user knows what is causing HDD thrashing / network accesses / CPU or memory load / etc.

Also, as Enki can now scan unexpected things, it should be prepared to handle malicious input (malformed, too long filenames, bad replies from syscalls for particular directories), i.e. /tmp/evil_directory/file_that_shall_not_be_scanned factor.

[andreikop]

Config option in the Settings dialogue will not solve the problem. We will have 2 bad modes. I also use Enki as an IDE and as a text editor. And if I'm editing some small file at /etc/ or ssh-fs, I would not like to have the file system scanned. But, if I already opened the file with enki /ssh/mount/point/the-file, it is too late to change the settings. I'm thinking about

• Implicit project open action
• @vi 's pseudocode
• scaning project on first FuzzyOpener use

[bjones1]

The simplest and most understandable solution to me seems to be the third option, scan on first FuzzyOpener use.

[vi]

Yes.

And closing Locator with unneeded FuzzyOpen attempt (i.e. if typed somefile instead of ./somefile) should stop scanning.

On the other hand, there should be a way to trigger background scanning for a large project. Most obvious is yet another Locator command like "scan" (there may be optional parameter to set project root explicitly). This explicit project root may be saved in the session.

[andreikop]

Have some time for enki again.

Now scanning is started only when fuzzy open is requested and cancelled when fuzzyopen is cancelled. But if enki managed to finish scanning the files - the list will be kept for the next time. Let's try to use this code and see if explicit scan command is necessary.

Current version has a bug and freezes often. I'll fix this bug soon.