YouCompleteMe with eigen3, incorrect syntax highlighting

[bayesianlab]

Issue Prelude

Please complete these steps and check these boxes (by putting an x inside the brackets) before filing your issue:

• [x ] I have read and understood YCM's [CONTRIBUTING][cont] document.
• [ x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
• [ x] I have read and understood YCM's [README][readme], especially the [Frequently Asked Questions][faq] section.
• [ x] I have searched YCM's issue tracker to find issues similar to the one I'm about to report and couldn't find an answer to my problem. ([Example Google search.][search])
• [ x] If filing a bug report, I have included the output of vim --version.
• [ x] If filing a bug report, I have included the output of :YcmDebugInfo.
• [ x] If filing a bug report, I have attached the contents of the logfiles using the :YcmToggleLogs command.
• [x ] If filing a bug report, I have included which OS (including specific OS version) I am using.
• [ x] If filing a bug report, I have included a minimal test case that reproduces my issue, including what I expected to happen and what actually happened.
• [ x] If filing a installation failure report, I have included the entire output of install.py (or cmake/make/ninja) including its invocation
• [ x] I understand this is an open-source project staffed by volunteers and that any help I receive is a selfless, heartfelt gift of their free time. I know I am not entitled to anything and will be polite and courteous.
• [x ] I understand my issue may be closed if it becomes obvious I didn't actually perform all of these steps.

I cannot get the syntax highlighting to work well for the Eigen library. I know about the .ycm_extra_conf.py file, I have used it and updated it before. I have gotten it to work properly in other contexts.

My eigen headers are located in /usr/local/include/eigen3. I changed the .ycm_extra_conf.py file to include these headers, I have also used the .json compilation database to try and help.

I created a small test case to illustrate the error,

#include <Eigen/Dense>

using namespace std;
using namespace Eigen;

int main(){
    MatrixXd IncorrectSyntax(10,4);
}

The syntax error says excess elements in scalar initializer. In my actual project I get this error and that Eigen::MatrixXd is not a structure or union, none of Eigen library seems to be recognized.

I have the .ycm_extra_conf.py file in the build directory of this project.

I only changed one part of the .ycm_extra_conf.py file to try and include eigen in this project.

There is no problem with this code, it compiles with CMake and runs without errors.

Note

I have seen this, https://github.com/Valloric/YouCompleteMe/issues/303, and the advice was to use '-isystem', '/usr/local/include/eigen3'. This does not do anything.

I am able to get my .ycm_extra_conf.py file to see my other headers in my include/ directory, but I am not able to get it to see eigen.

Diagnostic data

Output of vim --version

VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Jun 14 2016 16:06:49)
Compiled by root@apple.com
Normal version without GUI.  Features included (+) or not (-):
-arabic +autocmd -balloon_eval -browse +builtin_terms +byte_offset +cindent 
-clientserver -clipboard +cmdline_compl +cmdline_hist +cmdline_info +comments 
-conceal +cryptv +cscope +cursorbind +cursorshape +dialog_con +diff +digraphs 
-dnd -ebcdic -emacs_tags +eval +ex_extra +extra_search -farsi +file_in_path 
+find_in_path +float +folding -footer +fork() -gettext -hangul_input +iconv 
+insert_expand +jumplist -keymap -langmap +libcall +linebreak +lispindent 
+listcmds +localmap -lua +menu +mksession +modify_fname +mouse -mouseshape 
-mouse_dec -mouse_gpm -mouse_jsbterm -mouse_netterm -mouse_sysmouse 
+mouse_xterm +multi_byte +multi_lang -mzscheme +netbeans_intg -osfiletype 
+path_extra -perl +persistent_undo +postscript +printer -profile +python/dyn 
-python3 +quickfix +reltime -rightleft +ruby/dyn +scrollbind +signs 
+smartindent -sniff +startuptime +statusline -sun_workshop +syntax +tag_binary 
+tag_old_static -tag_any_white -tcl +terminfo +termresponse +textobjects +title
 -toolbar +user_commands +vertsplit +virtualedit +visual +visualextra +viminfo 
+vreplace +wildignore +wildmenu +windows +writebackup -X11 -xfontset -xim -xsmp
 -xterm_clipboard -xterm_save 
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
      user exrc file: "$HOME/.exrc"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -D_FORTIFY_SOURCE=0 -Iproto -DHAVE_CONFIG_H -arch i386 -arch x86_64 -g -Os -pipe
Linking: gcc -arch i386 -arch x86_64 -o vim -lncurses

Output of YcmDebugInfo

Printing YouCompleteMe debug information...
-- Client logfile: /var/folders/2x/40k5nsnj0vxf64l5gqjrd3080000gn/T/ycm_9uidQl.log
-- Server Python interpreter: //anaconda/bin/python
-- Server Python version: 2.7.12
-- Server has Clang support compiled in: True
-- Clang version: clang version 5.0.0 (tags/RELEASE_500/final)
-- Extra configuration file found and loaded
-- Extra configuration path: /Users/dillonflannery-valadez/Google Drive/CodeProjects/YCMTest/build/.ycm_extra_conf.py
-- C-family completer debug information:
--   Compilation database path: None
--   Flags: ['-Wall', '-Wextra', '-Werror', '-Wno-long-long', '-Wno-variadic-macros', '-fexceptions', '-DNDEBUG', '-x', 'c++', '-isystem', '/Users/dillonfl
annery-valadez/Google Drive/CodeProjects/YCMTest/BoostParts', '-isystem', '//anaconda/include/python2.7', '-isystem', '/Users/dillonflannery-valadez/Google
 Drive/CodeProjects/YCMTest/llvm/include', '-isystem', '/Users/dillonflannery-valadez/Google Drive/CodeProjects/YCMTest/llvm/tools/clang/include', '-I', '/
Users/dillonflannery-valadez/Google Drive/CodeProjects/YCMTest/build', '-I', '/Users/dillonflannery-valadez/Google Drive/CodeProjects/YCMTest/build/ClangCo
mpleter', '-isystem', '/Users/dillonflannery-valadez/Google Drive/CodeProjects/YCMTest/build/tests/gmock/gtest', '-isystem', '/Users/dillonflannery-valadez
/Google Drive/CodeProjects/YCMTest/build/tests/gmock/gtest/include', '-isystem', '/Users/dillonflannery-valadez/Google Drive/CodeProjects/YCMTest/build/tes
ts/gmock', '-isystem', '/Users/dillonflannery-valadez/Google Drive/CodeProjects/YCMTest/build/tests/gmock/include', '-isystem', '/Users/dillonflannery-vala
dez/Google Drive/CodeProjects/YCMTest/build/benchmarks/benchmark/include', '-I', '/usr/local/include/eigen3', '-resource-dir=/Users/dillonflannery-valadez/
.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../clang_includes', '-isystem', '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolc
hain/usr/include/c++/v1', '-isystem', '/Library/Developer/CommandLineTools/usr/include/c++/v1', '-isystem', '/usr/local/include', '-isystem', '/Application
s/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include', '-isystem', '/Library/Developer/CommandLineTools/usr/include', '-isystem',
 '/usr/include', '-isystem', '/System/Library/Frameworks', '-isystem', '/Library/Frameworks', '-isystem', '/Applications/Xcode.app/Contents/Developer/Toolc
hains/XcodeDefault.xctoolchain/usr/lib/clang/8.0.0/include', '-isystem', '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SD
Ks/MacOSX.sdk/usr/include', '-fspell-checking']
-- Server running at: http://127.0.0.1:64035
-- Server process ID: 21158
-- Server logfiles:
--   /var/folders/2x/40k5nsnj0vxf64l5gqjrd3080000gn/T/ycmd_64035_stdout_StVfez.log
--   /var/folders/2x/40k5nsnj0vxf64l5gqjrd3080000gn/T/ycmd_64035_stderr_OfSk4P.log
Press ENTER or type command to continue

OS version, distribution, etc.

OSX El Capitan 10.11.6

[micbou]

I can't reproduce. I installed the Eigen library with Homebrew and used the following .ycm_extra_conf.py:

flags = [
  '-isystem', '/usr/local/include/eigen3'
]

def FlagsForFile( filename ):
  return { 'flags': flags }

I got no errors from your test case. I see that your version of YCM is using Clang 5.0.0 while the current version is on Clang 6.0.0. Could you try with the latest version of YCM? You'll need to run again the install.py script.

[bayesianlab]

I reinstalled YCM and I still have the errors. I am including eigen as you did with the -isystem flag. I have errors all over the place. The version of Clang is now 6.0.0 I see from the debug info

[bayesianlab]

I tried micbou's simple .ycm_extra_conf.py file and installing eigen again with homebrew and I still have the same incorrect syntax highlighting.

[micbou]

Only difference between the configurations is the macOS version. I tried on 10.13. I am suspecting the Clang system headers on macOS 10.11 are not compatible with the Clang version downloaded by YCM (6.0.0). Are you still getting the errors if you build YCM with the --system-libclang flag?

./install.py --clang-completer --system-libclang

[bayesianlab]

micbou, Thank you for this suggestion!! I finally have gotten rid of the false errors by installing ycm with the --system-libclang flag in addition to the --clang-completer flag! This solved my problem, now I have correct recognition of the Eigen library with YCM. This has closed the issue for me.

Thanks!

[tgzhou98]

@dillon4287 I have the same problem, and I use clang 5.0.1 in Mac OSX 11.6. But when I recompile YCM with --system-libclang flag, the error is still here. So do I need to change the system clang to 5.0.0?

By the way, some of the identifier in the Eigen namepsace works correctly, but like Eigen::MatrixXd namespace work not well, how to solve that?

[bayesianlab]

I'm not sure why an older version of clang would help. Is your .ycm_extra_conf.py file right?

[tgzhou98]

Except the last line, everything is the default configuration of the ycm_extra_conf.py

An interesting phenomenon is, I have several times recognise the Eigen::MatrixXd, which means that I can use the function in the namespace. But a few seconds after, ycm raise the error. How strange!

2018-05-01 01:45:15,229 - DEBUG - Global extra conf not loaded or no function YcmCorePreload
2018-05-01 01:45:15,406 - INFO - Received ready request
2018-05-01 01:45:16,434 - INFO - Received event notification
2018-05-01 01:45:16,434 - DEBUG - Event name: BufferVisit
2018-05-01 01:45:16,468 - INFO - Received event notification
2018-05-01 01:45:16,469 - DEBUG - Event name: FileReadyToParse
2018-05-01 01:45:16,469 - INFO - Adding buffer identifiers for file: /Users/zhoutiangang/Program/C++/course/eigen.cpp
2018-05-01 01:45:19,515 - INFO - Received filetype completion available request
2018-05-01 01:45:20,982 - INFO - Received debug info request
2018-05-01 01:45:24,072 - INFO - Received event notification
2018-05-01 01:45:24,072 - DEBUG - Event name: BufferVisit
2018-05-01 01:45:24,072 - ERROR - No semantic completer exists for filetypes: ['txt']
Traceback (most recent call last):
  File "/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/server_state.py", line 100, in FiletypeCompletionAvailable
    self.GetFiletypeCompleter( filetypes )
  File "/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/server_state.py", line 89, in GetFiletypeCompleter
    current_filetypes ) )
ValueError: No semantic completer exists for filetypes: ['txt']

2018-05-01 01:46:37,121 - INFO - Received event notification
2018-05-01 01:46:37,122 - DEBUG - Event name: BufferVisit
2018-05-01 01:46:37,124 - INFO - Received event notification
2018-05-01 01:46:37,125 - DEBUG - Event name: FileReadyToParse
2018-05-01 01:46:37,125 - INFO - Adding buffer identifiers for file: /Users/zhoutiangang/Program/C++/course/eigen.cpp
2018-05-01 01:46:41,500 - INFO - Received completion request
2018-05-01 01:46:41,500 - DEBUG - Using filetype completion: False
2018-05-01 01:46:41,751 - INFO - Received event notification
2018-05-01 01:46:41,751 - DEBUG - Event name: CurrentIdentifierFinished
2018-05-01 01:46:41,751 - INFO - Adding ONE buffer identifier for file: /Users/zhoutiangang/Program/C++/course/eigen.cpp
2018-05-01 01:46:41,754 - INFO - Received completion request
2018-05-01 01:46:41,754 - DEBUG - Using filetype completion: False
2018-05-01 01:46:42,089 - INFO - Received completion request
2018-05-01 01:46:42,089 - DEBUG - Using filetype completion: True
2018-05-01 01:46:42,579 - INFO - Received event notification
2018-05-01 01:46:42,579 - DEBUG - Event name: CurrentIdentifierFinished
2018-05-01 01:46:42,580 - INFO - Adding ONE buffer identifier for file: /Users/zhoutiangang/Program/C++/course/eigen.cpp
2018-05-01 01:46:42,583 - INFO - Received completion request
2018-05-01 01:46:42,583 - DEBUG - Using filetype completion: False
2018-05-01 01:46:42,704 - INFO - Received completion request
2018-05-01 01:46:42,705 - DEBUG - Using filetype completion: True
2018-05-01 01:46:42,728 - ERROR - Exception from semantic completer (using general): Traceback (most recent call last):
  File "/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/handlers.py", line 103, in GetCompletions
    .ComputeCandidates( request_data ) )
  File "/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/completer.py", line 229, in ComputeCandidates
    candidates = self._GetCandidatesFromSubclass( request_data )
  File "/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/completer.py", line 240, in _GetCandidatesFromSubclass
    raw_completions = self.ComputeCandidatesInner( request_data )
  File "/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/cpp/clang_completer.py", line 165, in ComputeCandidatesInner
    raise RuntimeError( NO_COMPLETIONS_MESSAGE )
RuntimeError: No completions found; errors in the file?

This is stderr file

The 'txt' file type error may be contributed from open log file in gvim, which has nothing to do with the true error

[bayesianlab]

I'm not sure if this would help but have you included in your .ycm_extra_conf.py the actual source code and include directory? I think the person to really help you is @micbou because I do not know enough.

[micbou]

@VincentZhou98 Which version of macOS? What's the output of :YcmDebugInfo and :YcmDiags while editing your C++ file?

[bayesianlab]

OSX El Capitan 10.11.6

:YcmDiags gives no errors.

[tgzhou98]

@micbou The same, OSX EI Capitan 10.11.6

:YcmDebugInfo file

Printing YouCompleteMe debug information...
-- Client logfile: /var/folders/6n/0qbj_8gj6gz8psmtnypkz33c0000gn/T/ycm_3ZlTa4.log
-- Server Python interpreter: /usr/local/opt/python/bin/python3.6
-- Server Python version: 3.6.4
-- Server has Clang support compiled in: True
-- Clang version: clang version 6.0.0 (tags/RELEASE_600/final)
-- Extra configuration file found and loaded
-- Extra configuration path: /Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/ycm/.ycm_extra_conf.py
-- C-family completer debug information:
--   Compilation database path: None
--   Flags: ['-Wall', '-Wextra', '-Werror', '-Wno-long-long', '-Wno-variadic-macros', '-fexceptions', '-DNDEBUG', '-DUSE_CLANG_COMPLETER', '-DYCM_EXPORT=', '-x
', 'c++', '-isystem', '/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/pybind11', '-isystem', '/Users/zhoutiangang/.vim/bundle/YouCompleteMe
/third_party/ycmd/cpp/BoostParts', '-isystem', '/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/include/python3.6m', '-isystem', '/Us
ers/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/llvm/include', '-isystem', '/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/
llvm/tools/clang/include', '-I', '/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/ycm', '-I', '/Users/zhoutiangang/.vim/bundle/YouCompleteMe
/third_party/ycmd/cpp/ycm/ClangCompleter', '-isystem', '/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/ycm/tests/gmock/gtest', '-isystem',
'/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/ycm/tests/gmock/gtest/include', '-isystem', '/Users/zhoutiangang/.vim/bundle/YouCompleteMe/
third_party/ycmd/cpp/ycm/tests/gmock', '-isystem', '/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/ycm/tests/gmock/include', '-isystem', '/
Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/cpp/ycm/benchmarks/benchmark/include', '-isystem', '/usr/local/Cellar/eigen/3.3.4/include/eigen3'
, '-std=c++11', '-resource-dir=/Users/zhoutiangang/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../clang_includes', '-isystem', '/Applications/Xcode.app/Con
tents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1', '-isystem', '/usr/local/include', '-isystem', '/Applications/Xcode.app/Contents/Develo
per/Toolchains/XcodeDefault.xctoolchain/usr/include', '-isystem', '/usr/include', '-isystem', '/System/Library/Frameworks', '-isystem', '/Library/Frameworks',
'-isystem', '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/8.0.0/include', '-isystem', '/Applications/Xcode.app/
Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include', '-fspell-checking']
--   Translation unit: /Users/zhoutiangang/Program/C++/course/eigen.cpp
-- Server running at: http://127.0.0.1:54347
-- Server process ID: 96806
-- Server logfiles:
--   /var/folders/6n/0qbj_8gj6gz8psmtnypkz33c0000gn/T/ycmd_54347_stdout_apg2vo.log
--   /var/folders/6n/0qbj_8gj6gz8psmtnypkz33c0000gn/T/ycmd_54347_stderr_zwbjyc.log  

:YcmDiags

Woo, it looks like ycm look into the apple clang include directory! And the error about Eigen::MatrixXd is eigen.cpp|13 col 9 error| 'Eigen::Matrix2f' (aka 'int') is not a class, namespace, or enumeration

[bayesianlab]

Im at a loss as to why it isnt working for you. I would try to ask micbou

[tgzhou98]

I find when I include the <Eigen/Dense> file, the YcmDiags will have those error like /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/8.0.0/include/xmmintrin.h|646 col 3 error| use of undeclared identifier '__builtin_ia32_storeups'

These file may have conflict with Eigen, so how to solve it?

[puremourning]

Try building YCM with —system-libclang

[tgzhou98]

@puremourning You mean system-libclang is trying to use the apple-clang? I will try it.

[puremourning]

Yes. You need Apple clang for intrinsics on macOS.

[tgzhou98]

@puremourning Thanks. After using apple-clang, everything works like a charm. In conclusion, almost every version of the recent lib-clang ---- 5.0.0 5.0.1 6.0.0 with have conflict with system include files. In my remote Linux machine, the error doesn't occur.

[puremourning]

OSS libclangs are missing whatever special sauce Apple adds to make certain Xcode things work (like iOS)