gutentags not finding existing project tag files and is regenerating them every time I open a file in project

[timblaktu]

I have not been able to find any issues matching this basic problem. Tags are getting generated by gutentags where I have configured them to be generated, and vim is finding them and allowing me to use them for code navigation, however I am getting a "GTAGS not found" error every time I start vim, and both the ctags and gtags are getting re-generated everytime I start vim. My expectation is for gutentags to discover the existing tags and (a) not post the "GTAGS not found" error (which I realize may be a Gtags-cscope error), and (b) not invoke update_tags.sh on startup.

With this config, defined towards the end of my .vimrc:

let g:gutentags_modules = ['ctags', 'gtags_cscope']
let g:gutentags_project_root = ['.git', '.cproject']
let g:gutentags_add_default_project_roots=0
let g:gutentags_define_advanced_commands=1
let g:gutentags_cache_dir = expand('~/.cache/tags')
let g:gutentags_trace=1
augroup MyGutentagsStatusLineRefresher
    autocmd!
    autocmd User GutentagsUpdating call lightline#update()
    autocmd User GutentagsUpdated call lightline#update()
augroup END

let g:GtagsCscope_Auto_Map = 1
let g:GtagsCscope_Auto_Load = 1
"let g:GtagsCscope_Quiet = 1
set cscopetag
source ~/dotfiles/vim/.vim/gtags-cscope.vim

and with a projectroot set by the .cproject file at my project root:

[tblack-stretch]/home/tblack/src/lt > ls .cproject
.cproject

and ctags and gtags already created successfully by gutentags (and working within vim) :

[tblack-stretch]/home/tblack/src/lt > tree -sh ~/.cache/tags
/home/tblack/.cache/tags
├── [4.0K]  home-tblack-src-lt-
│   ├── [7.2M]  GPATH
│   ├── [182M]  GRTAGS
│   └── [1.2G]  GTAGS
└── [229M]  home-tblack-src-lt-tags

I get this when I open the root makefile in the project with vim (or any other file in the project):

[tblack-stretch]/home/tblack/src/lt > vim makefile
Gtags-cscope: GTAGS not found.
Press ENTER or type command to continue

Upon opening the makefile, my status line and the below trace output indicate that gutentags is regenerating both ctags and gtags files even though these already exist as shown above:

gutentags: Scanning buffer 'makefile' for gutentags setup...
gutentags: No specific project type.
gutentags: Adding cscope DB file: /home/tblack/.cache/tags/home-tblack-src-lt-/GTAGS
gutentags: Setting gutentags for buffer 'makefile'
gutentags: Generating tags file: /home/tblack/.cache/tags/home-tblack-src-lt-tags
gutentags: Running: ['/home/tblack/dotfiles/vim/.vim/pack/timblaktu/start/vim-gutentags/plat/unix/update_tags.sh', '-e', 'ctags', '-t', '/home/tblack/.cache/tags/home-tblack-src-lt-tags', '-p', '/home/tblack/src/lt', '-o', '/home/tblack/dot
files/vim/.vim/pack/timblaktu/start/vim-gutentags/res/ctags_recursive.options', '-l', '/home/tblack/.cache/tags/home-tblack-src-lt-tags.log']
gutentags: In:      /home/tblack/src/lt
gutentags: Generating tags file: /home/tblack/.cache/tags/home-tblack-src-lt-/GTAGS
gutentags: Running: ['gtags', '--incremental', '/home/tblack/.cache/tags/home-tblack-src-lt-']
gutentags: In:      /home/tblack/src/lt
gutentags:
Press ENTER or type command to continue
gutentags: Locking tags file...
gutentags: Running ctags on whole project
gutentags: ctags -f "/home/tblack/.cache/tags/home-tblack-src-lt-tags.temp"  --options=/home/tblack/dotfiles/vim/.vim/pack/timblaktu/start/vim-gutentags/res/ctags_recursive.options "/home/tblack/src/lt"
gutentags: Finished gtags_cscope job.
gutentags: Adding cscope DB file: /home/tblack/.cache/tags/home-tblack-src-lt-/GTAGS
gutentags: Replacing tags file
gutentags: mv -f "/home/tblack/.cache/tags/home-tblack-src-lt-tags.temp" "/home/tblack/.cache/tags/home-tblack-src-lt-tags"
gutentags: Unlocking tags file...
gutentags: Done.
gutentags: Finished ctags job.

I'm on debian 9, vim 8 (nogui), universal-ctags version 0.0.0(6418097e), gnu-global version 6.6.2, gutentags master:

[tblack-stretch]pack/timblaktu/start/vim-gutentags > git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
[tblack-stretch]pack/timblaktu/start/vim-gutentags > git rev-parse master
327bd979dd480a42b6edf3205136ac08cf349e8e

[ludovicchabant]

Hi! I'm not super familiar with the cscope side of things (half of the code was contributed by other people for that module).

Does the GTAGS file exist prior to opening Vim? If not, could it be that Gtags-cscope tries to add it, but it doesn't exist, and it only gets created by Gutentags a second later?

[timblaktu]

@ludovicchabant yes, the entire tree of files in ~/.cache/tags as I posted above exists before I open a file in that project in vim. Note that this is not just a GTAGS issue - the ctags file also exists beforehand, yet gutentags still creates it.

Is there some test I could do to help shed more light on this? Thanks.

[ludovicchabant]

Ok, so you could try to first restrict Gutentags' work to ctags (with let g:gutentags_modules = ['ctags']) so we can see if that works as expected for that.

Note that it's expected for Gutentags to rebuild the files/DBs/etc. on open by default (see g:gutentags_generate_on_new). You could try changing that setting as a test to see if it's indeed some kind of problem between Vim trying to read the GTAGS file at the same time it's being written to.... (comparatively, the ctags module doesn't write directly to the tags file, it writes to a .temp file and then switches the 2... maybe the cscope module should do the same if that's the problem).

[timblaktu]

When I let g:gutentags_modules = ['ctags']:

1. I still get the GTAGS not found Gtags-cscope error.
2. ctags still generate when vim opens the file.

When I let g:gutentags_generate_on_new=0, as you said the ctags no longer generate when vim opens. I see now that the behavior I want requires me to let gutentags_generate_on_missing=1.

So now the only mystery is 1. above, why the GTAGS not found Gtags-cscope error is happening...

I can see that the gtags-cscope.vim plugin is emitting this text within this function which is called when the plugin starts up:

function! s:GtagsCscope_GtagsRoot()
    let cmd = s:global_command . " -pq"
    let cmd_output = system(cmd)
    if v:shell_error != 0
        if v:shell_error == 3
            call s:Error('GTAGS not found.')
        else
            call s:Error('global command failed. command line: ' . cmd)
        endif
        return ''
    endif
    return strpart(cmd_output, 0, strlen(cmd_output) - 1)
endfunction

where

let s:global_command = $GTAGSGLOBAL
if s:global_command == ''
        let s:global_command = "global"
endif

When this code fails, it basically exits the main GtagsCscope() plugin function, so the plugin functionality isn't working.

Testing this manually in a shell:

[tblack-stretch]/home/tblack/src > which global
/usr/local/bin/global
[tblack-stretch]/home/tblack/src > echo $GTAGSGLOBAL

[tblack-stretch]/home/tblack/src > global -pq
[tblack-stretch]/home/tblack/src > echo $?
3

indicates the error is caused by gnu global -pq (which is supposed to simply print out the location of the GTAGS tags database) is returning an error 3 bc it cannot find it.

If I cd into the directory that contains the GTAGS db file, and rerun the command, it succeeds:

[tblack-stretch]/home/tblack/src > cd ~/.cache/tags/home-tblack-src-lt-
[tblack-stretch]tblack/.cache/tags/home-tblack-src-lt- > ls
GPATH  GRTAGS  GTAGS
[tblack-stretch]tblack/.cache/tags/home-tblack-src-lt- > global -pq
/home/tblack/.cache/tags/home-tblack-src-lt-
[tblack-stretch]tblack/.cache/tags/home-tblack-src-lt- > echo $?
0

So, it appears the problem I am having here is that gnu global needs to be configured to know my g:gutentags_cache_dir so that at startup it can find the GTAGS database.

Can anyone using gutentags with gnu global chime in on how I need to properly configure so that gnu global is able to dynamically find the GTAGS databases that gutentags is generating?

[timblaktu]

In my vimrc I am sourcing this file:

source ~/dotfiles/vim/.vim/gtags-cscope.vim

which is a copy of the gtags-cscope.vim plugin from the gnu global std distribution.

I see now that gutentags also ships with some gtags-cscope vimscript, in vim-gutentags/autoload/gutentags/gtags_cscope.vim, which does a lot of fancy things, including what looks like setting up the path to the gtags database files (by calling gutentags#get_cachefile)...

.. so, this sort of answers my question - I can see now that there is gutentags code in place that is supposed to be setting up the path to gtags database, dynamically, per-buffer.

I guess this means that I should turn off autoloading in MY gtags_cscope plugin:

let g:GtagsCscope_Auto_Load = 0

so that it will just let gutentags' gtags-cscope support do this instead?

Of course, when I do this, the error I was seeing goes away, but I'm not yet convinced that gutentags/gtags-cscope support is working, bc I always get "no matches found" when I invoke the standard cscope queries like :cs find g name, even though I see that the GTAGS for the corresponding project are there in my cache dir.

[ludovicchabant]

Yeah the gtags_cscope is enabled if you go back to having g:gutentags_modules contain it. Maybe the problems you had were both that and GtagsCscope conflicting each other?

Other than that, like I said, the gtags_cscope support in Gutentags is mostly provided by other contributors, I'm not using it myself... so if it doesn't work you might have to troubleshoot it yourself, or see if someone else replies to this thread.

[starifly]

mark