Autocomplete is sometimes slow on moderately sized (300 files) git repos

[paul-gauthier]

Issue

From NeverCast in discord:

NeverCast: I'm seeing a performance regression when typing /add, only 300 files, sometimes I'll get the file list hints come up, but most the time the terminal just halts

NeverCast: Add lrucache(1) to get tracked files, and lrucache(1,000) to normalize path seemed to help a lot, I haven't unit tested or end end tested that change, but it's usable now

NeverCast: Entirely possible though that this breaks the file list entirely, not checked that either

paulg: I think it's probably dangerous to simply cache these, because aider needs to react when the git repo changes. Purging the caches whever the user types their first character into a new chat input would probably get most benefits. And it's an easy cache eviction policy.

Version and model info

No response

[paul-gauthier]

https://discord.com/channels/1131200896827654144/1133060264825208942/1253171725315866668

Most of my aiderignores start with * and then I include the paths I want. Which is probably why ignores is such a hot path for me during tab completion

[myurko]

I have some pretty major performance issues with add on larger repos that I work with (10s of k of files). When I've profiled it, the get_tracked_files function seems to be called every character I type, can can take 10s of seconds (most of the time in normalize_path). Can we instead cache the tracked files? Adding files to the aiderignore doesn't help as that is post-normalization.

[bearjaws]

I have a legacy codebase with ~22k files in it, even paring it down to ~300 files it still is slow to search files using /add on a M2 MBP.

It definitely improves using an .aiderignore, as searching the full 22k files just freezes for 20+ seconds per character.

This is my .aiderignore for anyone else trying to work in a specific directory, this essentially ignores everything and then adds a few specific items back.

*

!.gitignore
!README.md
!/src/
!/src/**

At some point I might debug aider to see what it's slowing down on, 300 files is not a lot for a fuzzy search.