file_rec/git not obeying submodules

[tony]

Interesting file_rec/git issue.

vim config: https://github.com/tony/vim-config/blob/6d1f5f18c55367c33e6a230a1931db677d6cd8a9/settings/unite.vim

I have the following directories:

tony at yoga in ~ 
$ ls -al
drwxr-xr-x  13 tony tony    4096 Apr  1 02:15 .dot-config
lrwxrwxrwx   1 tony tony      27 Dec  6 10:58 .vim -> /home/tony/.dot-config/.vim
lrwxrwxrwx   1 tony tony      11 Dec 14 21:22 .vimrc -> .vim/.vimrc

~/.dot-config is https://github.com/tony/.dot-config. It's filled with git submodules. I clone this repository and symlink them to $HOME.

When I am inside of ~/.vim (which symlinks to ~/.dot-config/.vim, which is a git submodule), unite.vim is giving me the wrong files:

buffer       2 %a  No Name (06-04-2015 23:08:55)                                                                     
file_rec/git .editorconfig (.dot-config)           ~/.dot-config                                                     
file_rec/git .gitignore (.dot-config)              ~/.dot-config                                                     
file_rec/git .vimrc (.dot-config)                  ~/.dot-config                                                     
file_rec/git README.rst (.dot-config)              ~/.dot-config                                                     
file_rec/git TODO.rst (.dot-config)                ~/.dot-config                                                     
file_rec/git autocmd.vim (.dot-config)             ~/.dot-config                                                     
file_rec/git bundles.vim (.dot-config)             ~/.dot-config                                                     
file_rec/git colors.vim (.dot-config)              ~/.dot-config                                                     
file_rec/git bowerrc.vim (ftdetect)                ~/.dot-config/ftdetect                                            
file_rec/git jshintrc.vim (ftdetect)               ~/.dot-config/ftdetect                                            
file_rec/git watchmanconfig.vim (ftdetect)         ~/.dot-config/ftdetect                                            
file_rec/git functions.vim (.dot-config)           ~/.dot-config                                                     
file_rec/git ignore.vim (.dot-config)              ~/.dot-config                                                     
file_rec/git keymappings.vim (.dot-config)         ~/.dot-config                                                     
file_rec/git quickfix.vim (.dot-config)            ~/.dot-config                                                     
file_rec/git settings.vim (.dot-config)            ~/.dot-config                                                     
file_rec/git LaTeX-Box.vim (settings)              ~/.dot-config/settings                                            
file_rec/git NERDCommenter.vim (settings)          ~/.dot-config/settings                                            
file_rec/git NERDTree.vim (settings)               ~/.dot-config/settings                                            
file_rec/git airline.vim (settings)                ~/.dot-config/settings                                            
file_rec/git ctrlp.vim (settings)                  ~/.dot-config/settings                                            
file_rec/git delimitMate.vim (settings)            ~/.dot-config/settings                                            
file_rec/git easymotion.vim (settings)             ~/.dot-config/settings                                            
file_rec/git fugitive.vim (settings)               ~/.dot-config/settings                                            
file_rec/git gitgutter.vim (settings)              ~/.dot-config/settings                                            
file_rec/git hardtime.vim (settings)               ~/.dot-config/settings                                            
file_rec/git jedi.vim (settings)                   ~/.dot-config/settings                                            
file_rec/git jsbeautify.vim (settings)             ~/.dot-config/settings                                            
file_rec/git neocompleterc.vim (settings)          ~/.dot-config/settings                                            
file_rec/git pymode.vim (settings)                 ~/.dot-config/settings                                            
file_rec/git rainbow_parentheses.vim (settings)    ~/.dot-config/settings                                            
file_rec/git startify.vim (settings)               ~/.dot-config/settings                                            
file_rec/git supertab.vim (settings)               ~/.dot-config/settings                                            
file_rec/git syntastic.vim (settings)              ~/.dot-config/settings                                            
file_rec/git tagbar.vim (settings)                 ~/.dot-config/settings                                            
file_rec/git tcomment.vim (settings)               ~/.dot-config/settings                                            
file_rec/git ultisnips.vim (settings)              ~/.dot-config/settings                                            
file_rec/git unite.vim (settings)                  ~/.dot-config/settings                                            
file_rec/git vim-go.vim (settings)                 ~/.dot-config/settings                                            
file_rec/git vim-multiple-cursors.vim (settings)   ~/.dot-config/settings                                            
file_rec/git vim-nodejs-complete.vim (settings)    ~/.dot-config/settings                                            
file_rec/git vim-rooter.vim (settings)             ~/.dot-config/settings                                            
file_rec/git vimfiler.vim (settings)               ~/.dot-config/settings                                            
file_rec/git vimshell.vim (settings)               ~/.dot-config/settings                                            
file_rec/git youcompleteme.vim (settings)          ~/.dot-config/settings                                            
~                                                                                                                    
~                                                                                                                    
~                                                                                                                    
~                                                                                                                    
~                                                                                                                    
~                                                                                                                    
 buffer(1) file_rec/git(45)                                                   directory: /home/tony/.dot-config  1/47
[file_rec/git] directory: /home/tony/.dot-config                                                                     

Vim's pwd is /home/tony/.dot-config/.vim. The expected behavior would be to show /home/tony/.dot-config/.vim (the true path) or /home/tony/.vim (the symlinked directory I'm inside of).

$ vim --version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Apr  5 2015 13:16:22)
Included patches: 1-691
Compiled by tony@yoga
Huge version without GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +syntax
+arabic          +file_in_path    +mouse_sgr       +tag_binary
+autocmd         +find_in_path    -mouse_sysmouse  +tag_old_static
-balloon_eval    +float           +mouse_urxvt     -tag_any_white
-browse          +folding         +mouse_xterm     -tcl
++builtin_terms  -footer          +multi_byte      +terminfo
+byte_offset     +fork()          +multi_lang      +termresponse
+cindent         +gettext         -mzscheme        +textobjects
+clientserver    -hangul_input    +netbeans_intg   +title
+clipboard       +iconv           +path_extra      -toolbar
+cmdline_compl   +insert_expand   -perl            +user_commands
+cmdline_hist    +jumplist        +persistent_undo +vertsplit
+cmdline_info    +keymap          +postscript      +virtualedit
+comments        +langmap         +printer         +visual
+conceal         +libcall         +profile         +visualextra
+cryptv          +linebreak       +python          +viminfo
+cscope          +lispindent      -python3         +vreplace
+cursorbind      +listcmds        +quickfix        +wildignore
+cursorshape     +localmap        +reltime         +wildmenu
+dialog_con      +lua             +rightleft       +windows
+diff            +menu            -ruby            +writebackup
+digraphs        +mksession       +scrollbind      +X11
-dnd             +modify_fname    +signs           +xfontset
-ebcdic          +mouse           +smartindent     -xim
+emacs_tags      -mouseshape      -sniff           +xsmp_interact
+eval            +mouse_dec       +startuptime     +xterm_clipboard
+ex_extra        -mouse_gpm       +statusline      -xterm_save
+extra_search    -mouse_jsbterm   -sun_workshop    -xpm
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  fall-back for $VIM: "/usr/local/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H     -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1      
Linking: gcc   -L/usr/local/lib -Wl,--as-needed -o vim    -lSM -lICE -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lnsl   -ldl  -L/usr/lib -llua5.2  -L/usr/lib/python2.7/config -lpython2.7      

[Shougo]

Please check below result.

echo unite#util#substitute_path_separator(fnamemodify(finddir('.git', ';'), ':p:h:h'))
echo unite#util#substitute_path_separator(fnamemodify(finddir('.git', ';'), ':p'))

[tony]

echo unite#util#substitute_path_separator(fnamemodify(finddir('.git', ';'), ':p:h:h'))

/home/tony/.dot-config

echo unite#util#substitute_path_separator(fnamemodify(finddir('.git', ';'), ':p'))

/home/tony/.dot-config/.git

[tony]

echo pwd

/home/tony/.dot-config/.vim

[Shougo]

Please upload git ls-files --full-name result.

[tony]

tony at la in ~/.vim (master) 
$ git ls-files --full-name 
.editorconfig
.gitignore
.vimrc
README.rst
TODO.rst
autocmd.vim
bundles.vim
colors.vim
ftdetect/bowerrc.vim
ftdetect/jshintrc.vim
ftdetect/watchmanconfig.vim
functions.vim
ignore.vim
keymappings.vim
quickfix.vim
settings.vim
settings/LaTeX-Box.vim
settings/NERDCommenter.vim
settings/NERDTree.vim
settings/airline.vim
settings/ctrlp.vim
settings/delimitMate.vim
settings/easymotion.vim
settings/fugitive.vim
settings/gitgutter.vim
settings/hardtime.vim
settings/jedi.vim
settings/jsbeautify.vim
settings/neocompleterc.vim
settings/pymode.vim
settings/rainbow_parentheses.vim
settings/startify.vim
settings/supertab.vim
settings/syntastic.vim
settings/tagbar.vim
settings/tcomment.vim
settings/ultisnips.vim
settings/unite.vim
settings/vim-go.vim
settings/vim-multiple-cursors.vim
settings/vim-nodejs-complete.vim
settings/vim-rooter.vim
settings/vimfiler.vim
settings/vimshell.vim
settings/youcompleteme.vim

tony at la in ~/.vim (master) 
$ echo $PWD
/home/tony/.vim

[Shougo]

OK. I get it. It seems "git ls-files --full-name" bug.

       --full-name
           When run from a subdirectory, the command usually outputs paths relative to the current directory. This option
           forces paths to be output relative to the project top directory.

It does not work properly. I cannot fix it.

[Shougo]

"git ls-files" cannot detect symbolic links. You should report the problem in git upstream. It may be fixed in latest git.

[tony]

@Shougo I thought it was git's fault at first too.

This isn't a symlink issue - it's a submodule issue.

I think this has to do with unite.vim and how it handles' submodules, regardless of symbolic linking. git ls-files --full-name returns the same result in the real directory /home/tony/.dot-config/.vim/.

I think somehow vim or unite is going up a directory and concatenating the parent directory (the root git project).

/home/tony/.dot-config <- main git repo
/home/tony/.dot-config/.vim <- git submodule

These circumstances also happen when pwd is /home/tony/.dot-config/.vim:

echo unite#util#substitute_path_separator(fnamemodify(finddir('.git', ';'), ':p:h:h'))

/home/tony/.dot-config

echo unite#util#substitute_path_separator(fnamemodify(finddir('.git', ';'), ':p'))

/home/tony/.dot-config/.git/

[tony]

This is an issue to has to do with how .git directories are looked up.

If I do this, all works as expected:

mv ~/.vim ~/.vim.bak && mkdir ~/.testdir && cp -r ~/.dot-config/.vim ~/.testdir/.vim && ln -sf ~/.testdir/.vim ~/.vim

All works as expected.

Here's how I can replicate the problem:

mkdir ~/.testdir/.git

Then if I'm in ~/.testdir/.vim or ~/.vim, I get this:

[file_rec/git] directory: /home/tony/.testdir                                                                
[file_rec/async] fatal: Not a git repository: ../.git/modules/.vim

[Shougo]

OK. But I don't want to support git submodules. I tired... Pull request is wellcome.

[tony]

Thanks for your help so far!

Sleep and rest up @Shougo. Don't get burned out. We need to be strong.

[Shougo]

Then if I'm in ~/.testdir/.vim or ~/.vim, I get this:

This is feature.

To support git modules, you must parse .gitmodules file. You can do it.

[Shougo]

Sleep and rest up @Shougo. Don't get burned out. We need to be strong.

I have much tasks in to do list. I don't have time to implement it.

[Shougo]

I may fix the problem by not using --full-name.

[tony]

@Shougo : I was actually in the middle of looking at this for grabbing git dirs: https://github.com/tpope/vim-fugitive/blob/master/plugin/fugitive.vim#L129

echo fugitive#extract_git_dir("/home/tony/.dot-config/.vim/")

/home/tony/.dot-config/.git/modules/.vim

echo fugitive#extract_git_dir("/home/tony/.vim/")

/home/tony/.vim/../.git/modules/.vim < not sure if that would be ok.

[tony]

@Shougo I can confirm that fugitive's fugitive#extract_git_dir would probably work.

The case echo fugitive#extract_git_dir("/home/tony/.vim/") would never happen, since:

!echo pwd

of

/home/tony/.vim

would be a :pwd of /home/tony/.dot-config/.vim if the getcwd() was passed in.

Fugitive's code is licensed under the same license as VIM. I'm not sure if it's compatible with unite's license.

Are you still trying at this, or should I try to implement a fix?

[Shougo]

Please check it.

[tony]

@Shougo On it.

[tony]

@Shougo : It works. This is tremendously helpful.