libclang does not complete code the way 'clang' binary itself does

[Quuxplusone]

Bugzilla Link	PR40326
Status	NEW
Importance	P normal
Reported by	Denis Pronin (dannftk@yandex.ru)
Reported on	2019-01-15 13:50:37 -0800
Last modified on	2019-01-15 13:57:24 -0800
Version	6.0
Hardware	PC Linux
CC	klimek@google.com, llvm-bugs@lists.llvm.org, richard-llvm@metafoo.co.uk
Fixed by commit(s)
Attachments	project.tar.gz (12405 bytes, application/gzip) CMakeCache.txt (44595 bytes, text/plain) CMakeOutput.log (44946 bytes, text/plain)
Blocks
Blocked by
See also

Created attachment 21331
'project' and 'complete' programs

Hi,

 I faced an issue when diving into clang completion feature and trying
 to master it in gentoo. I am a C++ developer and my ligthweight IDE is
 sublime text 3, where I am trying to embed code completion feature
 (with EasyClangComplete plugin)

 My system: Linux dannftk-gentoo-host 4.18.10-gentoo-dannftk #1 SMP Mon Oct 1 12:54:09 MSK 2018 x86_64 Intel(R) Core(TM) i3-4005U CPU @ 1.70GHz GenuineIntel GNU/Linux

 My clang, llvm, libclang are versions of 6.0.1

 In gentoo everything is compiled from sources, so 'clang' is not an exception there

 I noticed that code completion quite well works in binary mode when
 'clang' is called explicitly by means of a command like that

     > clang -v -fsyntax-only -xc++ -Xclang -code-completion-
     at=project.cpp:5:9 project.cpp

 But it refuses to work with libclang, which is used to accelerate
 completion and get type-info required about types, variables, etc.

 I compiled a simple program where I call libclang's function by myself
 trying to obtain a list of completions with functions like
 'clang_parseTranslationUnit2', 'clang_reparseTranslationUnit',
 'clang_codeCompleteAt', but got only diagnostics messages about errors
 and missings and none completions. I started to look into it and found
 out that, when I work with libclang, it calls 'clang' in a different
 way so that 'clang' does not look into its default path
 '/usr/lib/clang/6.0.1/include' and because of this loses headers needed

 I searched for a work around and found it. If I provide to 'libclang'
 clang options like '-isystem /usr/include/c++/v1
 -i/usr/lib/clang/6.0.1/include' everything goes the way it should,
 completions appear

 Thus I conclude that, when libclang was being compiled, it somehow lost
 the built-in directory /usr/lib/clang/6.0.1/include

 I attached a project for you to try

 you need just untar project.tar.gz, then

     > cd project
     > mkdir build
     > cd build
     > cmake ../
     > cmake --build . --target complete

 Then try to execute the following command: this requested to complete
 the piece of code in project.cpp locating in a parent directory at 5th
 line and 9th column; -v is used to demonstrate how libclang calls clang
 and makes it verbose

     > ./complete ../project.cpp 5 9 -v

 I see the result, upon clang is done, like this:

     no type named 'const_pointer' in
     'std::__1::allocator_traits<std::__1::allocator<char> >'
     in instantiation of template class 'std::__1::basic_string<char,
     std::__1::char_traits<char>, std::__1::allocator<char> >' requested
     here
     no member named 'value' in 'std::__1::is_pod<char>'
     no member named 'value' in 'std::__1::is_same<char, char>'
     no member named 'value' in 'std::__1::is_same<char, char>'
     no member named 'value' in
     'std::__1::is_same<std::__1::basic_string<char,
     std::__1::char_traits<char>, std::__1::allocator<char> >::__rep,
     std::__1::allocator<char> >'
     in instantiation of template class
     'std::__1::__compressed_pair<std::__1::basic_string<char,
     std::__1::char_traits<char>, std::__1::allocator<char> >::__rep,
     std::__1::allocator<char> >' requested here
     in instantiation of template class 'std::__1::basic_string<char,
     std::__1::char_traits<char>, std::__1::allocator<char> >' requested
     here

 If you try to execute 'clang' itself avoiding featuring 'libclang' you
 will see well-done result

     > clang -fsyntax-only -xc++ -Xclang -code-completion-
     at=../project.cpp:5:9 -v ../project.cpp

 output:

     COMPLETION: append : [#basic_string<char> &#]append(<#const
     basic_string<char>
     &__str#>)

     COMPLETION: append : [#basic_string<char> &#]append(<#__self_view
     __sv#>)

     COMPLETION: append : [#basic_string<char> &#]append(<#const
     basic_string<char> &__str#>, <#size_type __pos#>{#, <#size_type __n
     = npos#>#})
     COMPLETION: append : [#typename
     enable_if<__can_be_converted_to_string_view<char, char_traits<char>,
     _Tp>::value, basic_string<char> &>::type#]append(<#const _Tp &__t#>,
     <#size_type __pos#>{#,
     <#size_type __n =
     npos#>#})

     COMPLETION: append : [#basic_string<char> &#]append(<#const
     value_type *__s#>, <#size_type
     __n#>)

     COMPLETION: append : [#basic_string<char> &#]append(<#const
     value_type
     *__s#>)

     COMPLETION: append : [#basic_string<char> &#]append(<#size_type
     __n#>, <#value_type
     __c#>)

     COMPLETION: append : [#typename
     enable_if<__is_exactly_input_iterator<_InputIterator>::value ||
     !__libcpp_string_gets_noexcept_iterator<_InputIterator>::value,
     basic_string<char> &>::type#]append(<#_InputIterator __first#>,
     <#_InputIterator __last#>)

     .....

 If you pay attention to what the include directories 'clang' examines
 to find a required include, you will notice that 'libclang' approach
 and 'clang' itself approach works differently

 if you try to force system headers in 'libclang' you will ses
 everything works properly. Beware, the order of system headers is meant
 to be preserved

     > ./complete ../project.cpp 5 9 -v -isystem /usr/include/c++/v1
     -isystem /usr/lib/clang/6.0.1/include

 Could you please sort it out why 'libclang' is compiled the way that
 'clang' default includes are lost?

PS: I left a comment in a github.com's project EasyClangComplete, please take a
look: https://github.com/niosus/EasyClangComplete/issues/282#issuecomment-
454154619

I attached programs 'complete' and 'project' to the issue, will also attach
CMake-specific files

Thank you

[Quuxplusone]

Attached project.tar.gz (12405 bytes, application/gzip): 'project' and 'complete' programs

[Quuxplusone]

Attached CMakeCache.txt (44595 bytes, text/plain): CMakeCache.txt

[Quuxplusone]

Attached CMakeOutput.log (44946 bytes, text/plain): CMakeOutput.log