Indexer should ignore specified paths

[johnplacek]

The indexer is running frequently and slowing down my machine when tests are creating ttl files in the build directory. The tests generate a large number of files and they are regenerated each time the tests run, each time triggering re-indexing. These files are ephemeral and primarily used for debugging failing tests. The files do not get committed to git and are superfluous once tests pass.

Would it be possible to exclude them from indexing? Some ideas:

1. indexer respects .gitignore and does not index ignored paths
2. add an explicit list of paths to lnkd config for the indexer to exclude

Thanks

[lunisolar]

I like this potential feature.

I've explored the possibilities and here are some thoughts:

.gitignore At first I remembered that parsing the git ignores is not an easy task - not that I parsed it perse. I remember that it can be in multiple places and potentially many instances of that file/config would need to be merged and also indexed. Thankfully IntelliJ does that already. And there is an API to access the VCS statuses and operations. It would cover potentially any VCS ignore list supported by the IDE.

exception list Git ignore has already build in support in IDE, actions like "add to ignore", "add to repository" and parsing of the all ignore files. So it seems that custom option would require significant work to be in par with reusing gitignore. On the other hand being marked for not indexing should also have independent ways to mark as ignored - other than VCS.

In overall, I think I will go, for now, with VCS-status and maybe in the feature as an addition the custom way (but I also expect that VCS-status will cover this nicely). Users will need to opt-in for this feature in settings.

I roughly see that this is something that can be included with couple of other performance improvements with next release (for now I expect it happen till the end of this month).

[johnplacek]

I like this potential feature.

I've explored the possibilities and here are some thoughts:

.gitignore At first I remembered that parsing the git ignores is not an easy task - not that I parsed it perse. I remember that it can be in multiple places and potentially many instances of that file/config would need to be merged and also indexed. Thankfully IntelliJ does that already. And there is an API to access the VCS statuses and operations. It would cover potentially any VCS ignore list supported by the IDE.

exception list Git ignore has already build in support in IDE, actions like "add to ignore", "add to repository" and parsing of the all ignore files. So it seems that custom option would require significant work to be in par with reusing gitignore. On the other hand being marked for not indexing should also have independent ways to mark as ignored - other than VCS.

In overall, I think I will go, for now, with VCS-status and maybe in the feature as an addition the custom way (but I also expect that VCS-status will cover this nicely). Users will need to opt-in for this feature in settings.

I roughly see that this is something that can be included with couple of other performance improvements with next release (for now I expect it happen till the end of this month).

Thanks. The intellij Excluded folder category seems to be appropriate for this:

Files in excluded folders are ignored by code completion, navigation and inspection.

https://www.jetbrains.com/help/idea/content-roots.html#folder-categories

[lunisolar]

The intellij Excluded folder

I checked this feature - having not use it for ages. It already works the expected way. Excluded there files/folders are not part of the indexing process. And actually the plugin do not even get to decide if those file should be indexed or not.

I use gradle/maven build engines and those are usually very well supported in intelliJ. However any project not using those build engines (nor any other engine supported by IntelliJ in auto-synchronized way) could still benefit from the option to check the VCS ignored files ()not every IDE has even the project structure dialog).

[lunisolar]

Upon further testing it seems that "intellij Excluded folder" might be the only way to do this properly.

Fun Fact 1: You can share files between projects - in the "Project Structure" ddialog ("PyCharm and Webstorm has this dialog inside "Settings") - you can add folders from outside project. Probably used very rarely. Fun Fact 2: VCS status is not propagated when sharing files by means of Fun Fact 1. Fun Fact 2: IntelliJ indexes are global (within the same IDE) - e.g. file shared by the two projects will be indexed only once.

Implementing optional custom rule in plugin for ignoring files during indexing that are already ignored in VCS when your projects use "Fun Fact 1" feature would cause files to be sometimes indexed, sometimes not (assuming overlap of VCS ignored files).

It seems that OP is happy with folder/file exclusion already build in IntelliJ.

Others, that reacted in the original post, do you still wait for the VCS ignore rules to influence indexing in the plugin? ( I'm on the fence with it).

[somanythings]

I'm in favour of the intellij excluded folder. But full disclosure, I nudged @johnplacek to suggest it 😛 Really like your feedback and call for preferences style! Hugely appreciated. Is there anyway we can help you with this?

[johnplacek]

It already works the expected way. Excluded there files/folders are not part of the indexing process.

I can confirm you are correct. The issue was we hadn't linked the gradle project so intellij was not automatically marking the /build directory as Excluded.

Thanks for your help - and sorry for wasting your time!

[lunisolar]

No worries. I actually decided to finish the implementation. And the option (inactive by default) to not index files ignored in VCS will be on next release.

I actually have opportunity to test indexing deeper, and right now I'm chasing rare cases of errors and optimize few things.