rdm reindexes more than necessary during common git workflows

[adembo]

During certain common git workflows (i.e. git rebase -i), rdm reindexes parts of the project even though logically, that may not be necessary. For example, consider the following sequence of events:

1. Commit two patches. The first modifies a test file, while the second modifies a central header file.
2. Realize you need to make a change to the test.
3. Commit a third patch with a small change to the test.
4. Run git rebase -i to apply the third patch as a fixup for the first patch.
5. Watch rdm reindex the world because the rebase caused the change to the central header file to be reverted, then reapplied.

This is especially acute for me because I'm running on a laptop (so this eats away precious battery life), and because I typically follow up a git rebase -i with a rebuild, which means my build process is competing for resources with rdm.

FWIW I'm running on Ubuntu 18 with rtags built from commit hash 015cde3b. I'm using clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final). I'm also using emacs 25.2.2, though I don't think that's relevant here.

I could disable automatic reindexing and control it myself out of emacs, but it'd be nice for rdm to automate the process. Here's a thought on how it can be done:

1. After reindexing a file, store a hash of its preprocessed output alongside it in the index.
2. Before reindexing it again, rehash the preprocessed output and only reindex if the hash differs from the hash stored in the index.
3. When a watch triggers for a file, delay the actual reindex by a few seconds, which should be enough time for something like git rebase -i to finish running (provided there's no more manual intervention necessary).
4. Alternatively, instead of delaying the reindex, allow the next watch trigger to cancel the in-progress reindex and start over. In the git rebase -i example above, the second watch trigger should yield a file with the same hashed contents as what's in the index, so reindexing will be skipped.

Another idea is to allow rdm to store multiple copies of a given file's index. This is similar to ccache, and perhaps was implemented in #234? Then, when reverting and applying the same change over and over, rdm will just switch between the two index copies instead of reindexing repeatedly. This could also help when switching branches (another common workflow).

[mckellyln]

@adembo I agree, especially about the switching branches part. If a hash of the input source was stored with its index - then when watcher sees file 'changed', hash input again and check against stored hash to see if it really is changed to prevent extra re-indexing.

[zhihuiFan]

hi @adembo I don't think I can help on this issue directly but I'd like to share some my experience.

a). Keep the .h file changes as small as possible. not just for rdm, it is also help for compiling. the method may be putting all the implementation to .cc file. b). disable the auto index when switch branch. I use the following aliases often.

alias freeze='rc --suspend all'
alias unfreeze='rc --suspend clear'

Like the 'freeze; make/git ..; unfreeze' style.

c). when I'm coding, I clear all the suspend files and run rdm -j 2 (looks if I suspend the index, the error check will not work, I want this feature so I have to let the index process running). I have 4 cpus, so 2 job just take 50% cpus. just let it run reindex slowly.

d). when I left my computer, I Ctl + c rdm and restart it with rdm (without the -j), it will use all the CPUs and reindex aggressively. I run rdm in terminal + tmux, so that I can restart it quickly.

e). I used helm-ag to search code when the navigation can't work well. Helm-ag also helps a lot in my experience.

[Andersbakken]

I have attempted to do something like this in the past without being particularly successful though I can't immediately remember what the issue was. I don't think I'd want to wait a couple of seconds before reindexing files though. I feel like this would be detrimental to the normal editing workflow.

I suppose something along the lines of zhihuiFan's approach of suspending rtags could be done in a git wrapper.

[adembo]

I don't think I'd want to wait a couple of seconds before reindexing files though. I feel like this would be detrimental to the normal editing workflow.

Fair enough. What about the alternative I suggested (see item 4 from the list I wrote)?