ycm-core / YouCompleteMe

A code-completion engine for Vim
http://ycm-core.github.io/YouCompleteMe/
GNU General Public License v3.0
25.45k stars 2.81k forks source link

libclang.so.8 not found when using ycm_global_extra_conf #3429

Closed rmrao closed 5 years ago

rmrao commented 5 years ago

Issue Prelude

Please complete these steps and check these boxes (by putting an x inside the brackets) before filing your issue:

Thank you for adhering to this process! It ensures your issue is resolved quickly and that neither your nor our time is needlessly wasted.

Issue Details

I've been attempting to use ycm_global_ycm_extra_conf. It looks like when doing so, there's an issue with importing ycm_core. I was able to reproduce this issue even when using ycmd's own .ycm_extra_conf.py.

Case - No Global Conf

Open any file under YouCompleteMe/third_party/ycmd/cpp/ycm/.

:YcmDebugInfo

Printing YouCompleteMe debug information...
-- Client logfile: /tmp/ycm_zp72q58g.log
-- Server Python interpreter: /home/rmrao/anaconda3/bin/python
-- Server Python version: 3.6.8
-- Server has Clang support compiled in: True
-- Clang version: clang version 8.0.0 (tags/RELEASE_800/final)
-- Extra configuration file found and loaded
-- Extra configuration path: /home/rmrao/thirdparty/ycmd/.ycm_extra_conf.py
-- C-family completer debug information:
--   Compilation database path: None
--   Flags: ['-Wall', '-Wextra', '-Werror', '-Wno-long-long', '-Wno-variadic-macros', '-fexceptions', '-DNDEBUG', '-DUSE_CLANG_COMPLETER', '-DYCM_EXPORT=', '-x', 'c++', '-isystem', '/home/rmrao/thirdparty/ycm
d/cpp/pybind11', '-isystem', '/home/rmrao/thirdparty/ycmd/cpp/whereami', '-isystem', '/home/rmrao/thirdparty/ycmd/cpp/BoostParts', '-isystem', '/home/rmrao/anaconda3/include/python3.6m', '-isystem', '/home/rm
rao/thirdparty/ycmd/cpp/llvm/include', '-isystem', '/home/rmrao/thirdparty/ycmd/cpp/llvm/tools/clang/include', '-I', '/home/rmrao/thirdparty/ycmd/cpp/ycm', '-I', '/home/rmrao/thirdparty/ycmd/cpp/ycm/ClangComp
leter', '-isystem', '/home/rmrao/thirdparty/ycmd/cpp/ycm/tests/gmock/gtest', '-isystem', '/home/rmrao/thirdparty/ycmd/cpp/ycm/tests/gmock/gtest/include', '-isystem', '/home/rmrao/thirdparty/ycmd/cpp/ycm/tests
/gmock', '-isystem', '/home/rmrao/thirdparty/ycmd/cpp/ycm/tests/gmock/include', '-isystem', '/home/rmrao/thirdparty/ycmd/cpp/ycm/benchmarks/benchmark/include', '-std=c++11', '-resource-dir=/home/rmrao/.vim/bu
ndle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/8.0.0', '-fspell-checking']
--   Translation unit: /home/rmrao/thirdparty/ycmd/cpp/ycm/Candidate.cpp
-- Server running at: http://127.0.0.1:55403
-- Server process ID: 31333
-- Server logfiles:
--   /tmp/ycmd_55403_stdout_fu17zivy.log
--   /tmp/ycmd_55403_stderr_ehon6wch.log

cat /tmp/ycmd_55403_stderr_ehon6wch.log

2019-06-29 16:30:13,111 - INFO - Received ready request
2019-06-29 16:30:13,160 - INFO - Received event notification
2019-06-29 16:30:13,162 - INFO - Received event notification
2019-06-29 16:30:13,163 - INFO - Adding buffer identifiers for file: /home/rmrao/thirdparty/ycmd/cpp/ycm/Candidate.cpp
2019-06-29 16:30:13,178 - INFO - No Clangd executable found in /home/rmrao/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clangd/output/bin
2019-06-29 16:30:13,561 - INFO - Received filetype completion available request
2019-06-29 16:30:15,842 - INFO - Received debug info request

Case 2 - Global Conf = '${HOME}/.vim/.ycm_extra_conf.py'

  1. mv .ycm_extra_conf.py ~/.vim
  2. add let g:ycm_global_ycm_extra_conf = ~/.vim/.ycm_extra_conf.py to .vimrc
  3. Open same file

:YcmDebugInfo

Printing YouCompleteMe debug information...
-- Client logfile: /tmp/ycm_mc_6uuci.log
-- Server Python interpreter: /home/rmrao/anaconda3/bin/python
-- Server Python version: 3.6.8
-- Server has Clang support compiled in: True
-- Clang version: clang version 8.0.0 (tags/RELEASE_800/final)
-- Extra configuration file found and loaded
-- Extra configuration path: /home/rmrao/.vim/.ycm_extra_conf.py
-- C-family completer debug information:
--   Compilation database path: None
--   Flags: ['-Wall', '-Wextra', '-Werror', '-Wno-long-long', '-Wno-variadic-macros', '-fexceptions', '-DNDEBUG', '-DUSE_CLANG_COMPLETER', '-DYCM_EXPORT=', '-x', 'c++', '-isystem', '/home/rmrao/.vim/cpp/pybin
d11', '-isystem', '/home/rmrao/.vim/cpp/whereami', '-isystem', '/home/rmrao/.vim/cpp/BoostParts', '-isystem', '/home/rmrao/anaconda3/include/python3.6m', '-isystem', '/home/rmrao/.vim/cpp/llvm/include', '-isy
stem', '/home/rmrao/.vim/cpp/llvm/tools/clang/include', '-I', '/home/rmrao/.vim/cpp/ycm', '-I', '/home/rmrao/.vim/cpp/ycm/ClangCompleter', '-isystem', '/home/rmrao/.vim/cpp/ycm/tests/gmock/gtest', '-isystem',
 '/home/rmrao/.vim/cpp/ycm/tests/gmock/gtest/include', '-isystem', '/home/rmrao/.vim/cpp/ycm/tests/gmock', '-isystem', '/home/rmrao/.vim/cpp/ycm/tests/gmock/include', '-isystem', '/home/rmrao/.vim/cpp/ycm/ben
chmarks/benchmark/include', '-std=c++11', '-resource-dir=/home/rmrao/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/8.0.0', '-fspell-checking']
--   Translation unit: /home/rmrao/thirdparty/ycmd/cpp/ycm/Candidate.cpp
-- Server running at: http://127.0.0.1:42243
-- Server process ID: 758
-- Server logfiles:
--   /tmp/ycmd_42243_stdout_mlmttqdh.log
--   /tmp/ycmd_42243_stderr_fmxhmeyl.log

cat /tmp/ycmd_42243_stderr_fmxhmeyl.log

Traceback (most recent call last):
  File "/home/rmrao/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/extra_conf_store.py", line 99, in _CallGlobalExtraConfMethod
    module = Load( global_ycm_extra_conf, force = True )
  File "/home/rmrao/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/extra_conf_store.py", line 178, in Load
    module = LoadPythonSource( _RandomName(), module_file )
  File "/home/rmrao/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/utils.py", line 517, in LoadPythonSource
    return importlib.machinery.SourceFileLoader( name, pathname ).load_module()
  File "<frozen importlib._bootstrap_external>", line 399, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 823, in load_module
  File "<frozen importlib._bootstrap_external>", line 682, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/rmrao/.vim/.ycm_extra_conf.py", line 35, in <module>
    import ycm_core
ImportError: libclang.so.8: cannot open shared object file: No such file or directory
2019-06-29 16:36:16,917 - INFO - Received ready request
2019-06-29 16:36:16,962 - INFO - Received event notification
2019-06-29 16:36:16,965 - INFO - Received event notification
2019-06-29 16:36:16,965 - INFO - Adding buffer identifiers for file: /home/rmrao/thirdparty/ycmd/cpp/ycm/Candidate.cpp
2019-06-29 16:36:16,977 - INFO - No Clangd executable found in /home/rmrao/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clangd/output/bin
2019-06-29 16:36:17,366 - INFO - Received filetype completion available request
2019-06-29 16:36:19,405 - INFO - Received debug info request

If I had to guess, the global conf is loaded before YCM updates the system path to include LIBCLANG_DIR, which causes the failure (just a guess based on the error and the fact that LIBCLANG_DIR does have to be added to the path I think?).

Diagnostic data

Output of vim --version

VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Jun 06 2019 17:31:41)
Included patches: 1-1453
Modified by pkg-vim-maintainers@lists.alioth.debian.org
Compiled by pkg-vim-maintainers@lists.alioth.debian.org
Huge version without GUI.  Features included (+) or not (-):
+acl               +farsi             +mouse_sgr         -tag_any_white
+arabic            +file_in_path      -mouse_sysmouse    -tcl
+autocmd           +find_in_path      +mouse_urxvt       +termguicolors
-autoservername    +float             +mouse_xterm       +terminal
-balloon_eval      +folding           +multi_byte        +terminfo
+balloon_eval_term -footer            +multi_lang        +termresponse
-browse            +fork()            -mzscheme          +textobjects
++builtin_terms    +gettext           +netbeans_intg     +timers
+byte_offset       -hangul_input      +num64             +title
+channel           +iconv             +packages          -toolbar
+cindent           +insert_expand     +path_extra        +user_commands
-clientserver      +job               -perl              +vertsplit
-clipboard         +jumplist          +persistent_undo   +virtualedit
+cmdline_compl     +keymap            +postscript        +visual
+cmdline_hist      +lambda            +printer           +visualextra
+cmdline_info      +langmap           +profile           +viminfo
+comments          +libcall           -python            +vreplace
+conceal           +linebreak         +python3           +wildignore
+cryptv            +lispindent        +quickfix          +wildmenu
+cscope            +listcmds          +reltime           +windows
+cursorbind        +localmap          +rightleft         +writebackup
+cursorshape       -lua               -ruby              -X11
+dialog_con        +menu              +scrollbind        -xfontset
+diff              +mksession         +signs             -xim
+digraphs          +modify_fname      +smartindent       -xpm
-dnd               +mouse             +startuptime       -xsmp
-ebcdic            -mouseshape        +statusline        -xterm_clipboard
+emacs_tags        +mouse_dec         -sun_workshop      -xterm_save
+eval              +mouse_gpm         +syntax
+ex_extra          -mouse_jsbterm     +tag_binary
+extra_search      +mouse_netterm     +tag_old_static
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H   -Wdate-time  -g -O2 -fdebug-prefix-map=/build/vim-8l5tlu/vim-8.0.1453=. -fstack-protector-strong -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY
_SOURCE=1
Linking: gcc   -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim        -lm -ltinfo -lnsl  -lselinux  -lacl -lattr -lgpm -ldl     -L/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu -lpyth
on3.6m -lpthread -ldl -lutil -lm

Contents of YCM, ycmd and completion engine logfiles

Included above

OS version, distribution, etc.

Ubuntu 18.04.

bstaletic commented 5 years ago

This is surprising. Can you show us what's in your global extra conf? Are you doing anything that requires ycm_core to be loaded in your extra conf?

rmrao commented 5 years ago

So I'm relatively new to using YCM with clang completion - I mostly do python development, so it's very possible I'm messing something up, although I tried replicating numerous times before submitting an issue.

To get up and running, I was trying the conf file from here. I had been able to replicate this issue though with both the ycmd extra conf and various conf files from several other projects (that aren't developed by me). See here and here (latter requires an academic license). I was also able to replicate this issue on a different machine (Ubuntu 16.04 and an older version of YCM that used libclang 7.0).

On the other hand, I just signed in to do some editing today and this doesn't seem to actually affect anything in terms of completion? At least, a lot of the basic completion commands like Goto are working.

rmrao commented 5 years ago

Just to follow up with some info I saw requested in a previous issue:

ldd ycm_core.so:

        linux-vdso.so.1 (0x00007ffc053a0000)
        libpython3.6m.so.1.0 => /home/rmrao/anaconda3/lib/libpython3.6m.so.1.0 (0x00007ffb02ada000)
        libclang.so.8 => not found
        libstdc++.so.6 => /home/rmrao/anaconda3/lib/libstdc++.so.6 (0x00007ffb037ec000)
        libgcc_s.so.1 => /home/rmrao/anaconda3/lib/libgcc_s.so.1 (0x00007ffb037d8000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffb026e9000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ffb024ca000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ffb022c6000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007ffb020c3000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007ffb01ebb000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ffb01b1d000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ffb0375d000)

objdump -x ycm_core.so | grep RPATH

^no output generated. Not clear if this is expected or not.

puremourning commented 5 years ago

So I think what's happening is that we're calling the global extra conf ( which imports ycm_core itself ) before loading ycm_core ourselves, and the global extra conf is also trying to load ycm_core.

This is not cool. It is not legitimate for global extra confs to import ycm_core in the global scope. The reason for this is a (legacy, undocumented) feature where global extra conf files can hook a "ycm core preload" hook which allows them to e.g. pre-load certain dlls (like, custom libclangs or something) before ycm_core.so is loaded into ycmd.

I don't think I would describe the linked extra conf as "better" personally, but you can probably fix it by moving the import to within the FlagsForFile (sic: should now be Settings) function.

Anyway, I would strongly recommend you don't copy/paste someone else extra conf file. Prefer to read the docs and either write your own or use a compilation database. For most people the later is easier, but the former should be a walk in the park for a python developer. All your Settings function has to do is return a dict with a single flags key which is a list of the compiler flags to use.

I think the problem would not happen with a trivial global extra conf such as the one in the docs:

def Settings( **kwargs ):
  return { 'flags': [ '-x', 'c++' ] }

Therefore I'm closing this. We can re-open if one of the following doesn't solve it"

akshit-sharma commented 5 years ago

I am having the same issue. I can open a separate issue and am just looking for some guidance on how to approach it.

  1. I want to use global extra conf because then I can just make a new small program quickly and run them (for testing a small concept). Making cmake would be an additional step. I would mind it but I would prefer if I can get global extra conf to work for a single file if possible. Similarly, having a local copy for extra conf will mean an extra copy each time, I make a new folder to test something simple.
  2. Without using ycm_core, it is hard to use CompilationDatabase to get a database if it exists. I use cmake as my build system for my projects. I want to be able to extract compilation_information compile_commands.json for such projects.
  3. Having a Settings function would mean FlagsForFile would not be called. (flags.py on line 278, has the first preference is Settings and if its not found then FlagsForFile). Also, I want to work with at least two kinds of file if possible (cpp and cu) and that why I want to have FlagsForFile. This will allow be to have flags specific to file type.

Problem, I want to tackle:

  1. Use compile db if its found in the directory I am working on. (without having another extra conf in the project if possible).
  2. I want to be able to compile single implementation files with minimum steps if possible. (Ideally, make a folder and create main.cpp/main.cu). With each file type having its own compilation flags.

I would like to know if it's possible to achieve the above without making any changes in the repository containing project?

Thank you

puremourning commented 5 years ago

Setting and FlagsForFile are equivalent. What I said was move the import to within the function so it happens after ycm core is properly loaded.

akshit-sharma commented 5 years ago

Oh, sorry. I didn't know it was possible to import inside a function. I also got confused between Setting and FlagsForFile because of different type of argument and overlooked the filename is also passed to Setting.

I tried having import ycm_core in starting function (Setting or FlagsForFile) but used it in a function called by starting function. So it wasn't found and I got confused. It just had to be in the same function which was using ycm_core.

Thanks for your time, I am able to get it working now.