search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28.66k stars 11.85k forks source link

clangd - Crash updating the AST #109037

Closed bgemmill closed 2 weeks ago

bgemmill commented 1 month ago

Clang 19 from the repository at https://apt.llvm.org

$ clang --version
Ubuntu clang version 19.1.0 (++20240903024228+0c641568515a-1~exp1~20240903144250.30)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-19/bin

Crash within clangd analyzing code:

PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
 #0 0x000075eb0e08abdf llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/llvm-19/bin/../lib/libLLVM.so.19.1+0xec6bdf)
 #1 0x000075eb0e0888e9 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-19/bin/../lib/libLLVM.so.19.1+0xec48e9)
 #2 0x000075eb0e08b330 (/usr/lib/llvm-19/bin/../lib/libLLVM.so.19.1+0xec7330)
 #3 0x000075eb0c321320 (/lib/x86_64-linux-gnu/libc.so.6+0x45320)
 #4 0x000075eb0e022598 llvm::StringMapImpl::LookupBucketFor(llvm::StringRef, unsigned int) (/usr/lib/llvm-19/bin/../lib/libLLVM.so.19.1+0xe5e598)
 #5 0x000062213975bf26 (/usr/bin/clangd+0x66cf26)
 #6 0x000062213975a408 (/usr/bin/clangd+0x66b408)
 #7 0x0000622139d7ac5f (/usr/bin/clangd+0xc8bc5f)
 #8 0x00006221394db096 (/usr/bin/clangd+0x3ec096)
 #9 0x000062213957d989 (/usr/bin/clangd+0x48e989)
#10 0x0000622139580d7d (/usr/bin/clangd+0x491d7d)
#11 0x0000622139579b90 (/usr/bin/clangd+0x48ab90)
#12 0x00006221395796eb (/usr/bin/clangd+0x48a6eb)
#13 0x00006221396e2841 (/usr/bin/clangd+0x5f3841)
#14 0x000075eb0c378a94 (/lib/x86_64-linux-gnu/libc.so.6+0x9ca94)
#15 0x000075eb0c405c3c (/lib/x86_64-linux-gnu/libc.so.6+0x129c3c)
Signalled during AST worker action: Update

Please let me know what else I should provide

llvmbot commented 1 month ago

@llvm/issue-subscribers-clangd

Author: Ben Gemmill (bgemmill)

Clang 19 from the repository at https://apt.llvm.org ``` $ clang --version Ubuntu clang version 19.1.0 (++20240903024228+0c641568515a-1~exp1~20240903144250.30) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/lib/llvm-19/bin ``` Crash within clangd analyzing code: ``` PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. #0 0x000075eb0e08abdf llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/llvm-19/bin/../lib/libLLVM.so.19.1+0xec6bdf) #1 0x000075eb0e0888e9 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-19/bin/../lib/libLLVM.so.19.1+0xec48e9) #2 0x000075eb0e08b330 (/usr/lib/llvm-19/bin/../lib/libLLVM.so.19.1+0xec7330) #3 0x000075eb0c321320 (/lib/x86_64-linux-gnu/libc.so.6+0x45320) #4 0x000075eb0e022598 llvm::StringMapImpl::LookupBucketFor(llvm::StringRef, unsigned int) (/usr/lib/llvm-19/bin/../lib/libLLVM.so.19.1+0xe5e598) #5 0x000062213975bf26 (/usr/bin/clangd+0x66cf26) #6 0x000062213975a408 (/usr/bin/clangd+0x66b408) #7 0x0000622139d7ac5f (/usr/bin/clangd+0xc8bc5f) #8 0x00006221394db096 (/usr/bin/clangd+0x3ec096) #9 0x000062213957d989 (/usr/bin/clangd+0x48e989) #10 0x0000622139580d7d (/usr/bin/clangd+0x491d7d) #11 0x0000622139579b90 (/usr/bin/clangd+0x48ab90) #12 0x00006221395796eb (/usr/bin/clangd+0x48a6eb) #13 0x00006221396e2841 (/usr/bin/clangd+0x5f3841) #14 0x000075eb0c378a94 (/lib/x86_64-linux-gnu/libc.so.6+0x9ca94) #15 0x000075eb0c405c3c (/lib/x86_64-linux-gnu/libc.so.6+0x129c3c) Signalled during AST worker action: Update ``` Please let me know what else I should provide
HighCommander4 commented 1 month ago

A symbolized stack trace is a good start, see https://clangd.llvm.org/troubleshooting#getting-stacktraces-from-crashes.

If you're able to provide a code example that reproduces the crash, that would be even better.

bgemmill commented 1 month ago

The symbolizer does not appear to be in the llvm apt repo, is there a good way to obtain it for the verison of llvm there?

$ apt show llvm-symbolizer*
N: Unable to locate package llvm-symbolizer*
N: Couldn't find any package by glob 'llvm-symbolizer*'
N: Unable to locate package llvm-symbolizer*
N: Couldn't find any package by glob 'llvm-symbolizer*'
E: No packages found
HighCommander4 commented 1 month ago

The symbolizer does not appear to be in the llvm apt repo, is there a good way to obtain it for the verison of llvm there?

The llvm-19 package should provide it.

vient commented 1 month ago

Wonder if llvm-symbolizer will help, binaries in llvm apt repo are stripped. Also note that 20240903024228 is an rc4 version - current version is 20240910033111, and 19.1.0-final packages are not uploaded yet.

bgemmill commented 1 month ago

Ok, the symbolizer is installed:

$ llvm-symbolizer --version
llvm-symbolizer
Ubuntu LLVM version 19.1.0
  Optimized build.

But per @vient crashes still seem to show the binary addresses only.

I'm going to see if I can isolate the file involved.

bgemmill commented 1 month ago

By copying all of the offending headers into a cpp, the crash no longer occurs, but clangd fails to ever complete parsing. This seems to affect programs as simple as int main() { return 0; } with no includes.

The last item in the log that looks interesting is: ASTWorker building file [path goes here]/clangd-crash.cpp version 2 with command [cmake output location]

But then clangd uses 100% cpu and never finishes. RES looks steady, it looks like it would run forever.

I'm running clangd with --compile-commands-dir=${workspaceFolder}/[cmake output location] --enable-config and cmake has set(CMAKE_EXPORT_COMPILE_COMMANDS 1) to feed it.

edit: workspaceFolder above is set by vscode, which in turn runs clangd.

HighCommander4 commented 1 month ago

By copying all of the offending headers into a cpp, the crash no longer occurs, but clangd fails to ever complete parsing. This seems to affect programs as simple as int main() { return 0; } with no includes.

This makes me wonder if the the hang is occurring while indexing standard library headers (clangd has a feature to index these in the background even if you don't include them, so that it can offer auto-complete suggestions even before you included a standard library header).

Can you try with a .clangd file in your workspace root containing:

Index:
  StandardLibrary: No
bgemmill commented 1 month ago

Having this in the workspace root does not seem to have an effect:

Index:
  StandardLibrary: No

While trying to reproduce, I noticed that the clang version in the apt repository changed to ++20240917083724+560ed047d183-1~exp1~20240917083901.36

Is the issue here that 19.1.0 just hasn't been released yet?

vient commented 1 month ago

19.1.0 just hasn't been released yet?

Yes, Debian CI seems to be very slow.

What distro are you on, Ubuntu 24.04? Ubuntu 20.04 still has no 19.1.0-final packages available.

bgemmill commented 1 month ago

It's a docker image from ubuntu 24.04.1:

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.04
DISTRIB_CODENAME=noble
DISTRIB_DESCRIPTION="Ubuntu 24.04.1 LTS"
HighCommander4 commented 1 month ago

Maybe once binary packages are posted to https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.0, using that will allow getting a useful stack trace.

Or if the crash occurs with clangd 18 as well, the Linux binary package posted to https://github.com/llvm/llvm-project/releases/tag/llvmorg-18.1.8 can be used.

bgemmill commented 1 month ago

18 from the llvm apt repository does not show this issue, at least this version:

$ clang --version
Ubuntu clang version 18.1.8 (++20240731025043+3b5b5c1ec4a3-1~exp1~20240731145144.92)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

I've moved back to that in the interim and I'll try 19 again when the release happens (or one of you nudges me to try again!)

Is there any way to make the symbolizer useful for the version in the apt repository? That version is considerably easier for me to test.

vient commented 1 month ago

Binaries in apt repo are stripped of debug information, nothing you can do - if I did not miss anything and there is no -dbg package or debuginfod server.

HighCommander4 commented 1 month ago

Is there any way to make the symbolizer useful for the version in the apt repository?

cc @sylvestre, as point of contact on the apt.llvm.org page, for this question.

sandymartel commented 1 month ago

Here's mine

 #0 0x00000001011663b0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/llvm/bin/clangd+0x1001823b0)
 #1 0x00000001011647c8 llvm::sys::RunSignalHandlers() (/opt/llvm/bin/clangd+0x1001807c8)
 #2 0x0000000101166a6c SignalHandler(int) (/opt/llvm/bin/clangd+0x100182a6c)
 #3 0x0000000196936584 (/usr/lib/system/libsystem_platform.dylib+0x18047a584)
 #4 0x0000000104024e10 std::__1::pair<llvm::StringMapIterator<llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>>, bool> llvm::StringMap<llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>, llvm::MallocAllocator>::try_emplace_with_hash<llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>&>(llvm::StringRef, unsigned int, llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>&) (/opt/llvm/bin/clangd+0x103040e10)
 #5 0x0000000104024e10 std::__1::pair<llvm::StringMapIterator<llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>>, bool> llvm::StringMap<llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>, llvm::MallocAllocator>::try_emplace_with_hash<llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>&>(llvm::StringRef, unsigned int, llvm::IntrusiveRefCntPtr<clang::tidy::utils::UseRangesCheck::Replacer>&) (/opt/llvm/bin/clangd+0x103040e10)
 #6 0x0000000104022dec clang::tidy::boost::UseRangesCheck::getReplacerMap() const (/opt/llvm/bin/clangd+0x10303edec)
 #7 0x00000001047acef0 clang::tidy::utils::UseRangesCheck::registerMatchers(clang::ast_matchers::MatchFinder*) (/opt/llvm/bin/clangd+0x1037c8ef0)
 #8 0x000000010234ed74 clang::clangd::ParsedAST::build(llvm::StringRef, clang::clangd::ParseInputs const&, std::__1::unique_ptr<clang::CompilerInvocation, std::__1::default_delete<clang::CompilerInvocation>>, llvm::ArrayRef<clang::clangd::Diag>, std::__1::shared_ptr<clang::clangd::PreambleData const>) (/opt/llvm/bin/clangd+0x10136ad74)
 #9 0x00000001023e057c clang::clangd::(anonymous namespace)::ASTWorker::generateDiagnostics(std::__1::unique_ptr<clang::CompilerInvocation, std::__1::default_delete<clang::CompilerInvocation>>, clang::clangd::ParseInputs, std::__1::vector<clang::clangd::Diag, std::__1::allocator<clang::clangd::Diag>>) (/opt/llvm/bin/clangd+0x1013fc57c)
#10 0x00000001023e0090 clang::clangd::(anonymous namespace)::ASTWorker::updatePreamble(std::__1::unique_ptr<clang::CompilerInvocation, std::__1::default_delete<clang::CompilerInvocation>>, clang::clangd::ParseInputs, std::__1::shared_ptr<clang::clangd::PreambleData const>, std::__1::vector<clang::clangd::Diag, std::__1::allocator<clang::clangd::Diag>>, clang::clangd::WantDiagnostics)::$_12::operator()() (/opt/llvm/bin/clangd+0x1013fc090)
#11 0x00000001023dcff0 clang::clangd::(anonymous namespace)::ASTWorker::runTask(llvm::StringRef, llvm::function_ref<void ()>) (/opt/llvm/bin/clangd+0x1013f8ff0)
#12 0x00000001023db848 void llvm::detail::UniqueFunctionBase<void>::CallImpl<clang::clangd::(anonymous namespace)::ASTWorker::create(llvm::StringRef, clang::clangd::GlobalCompilationDatabase const&, clang::clangd::TUScheduler::ASTCache&, clang::clangd::TUScheduler::HeaderIncluderCache&, clang::clangd::AsyncTaskRunner*, clang::clangd::Semaphore&, clang::clangd::TUScheduler::Options const&, clang::clangd::ParsingCallbacks&)::$_4>(void*) (/opt/llvm/bin/clangd+0x1013f7848)
#13 0x0000000102592984 void* llvm::thread::ThreadProxy<std::__1::tuple<clang::clangd::AsyncTaskRunner::runAsync(llvm::Twine const&, llvm::unique_function<void ()>)::$_4>>(void*) (/opt/llvm/bin/clangd+0x1015ae984)
#14 0x0000000196905f94 (/usr/lib/system/libsystem_pthread.dylib+0x180449f94)
#15 0x0000000196900d34 (/usr/lib/system/libsystem_pthread.dylib+0x180444d34)
Signalled during AST worker action: Build AST

built from source

> /opt/llvm/bin/clangd --version
clangd version 19.1.0 (https://github.com/llvm/llvm-project.git a4bf6cd7cfb1a1421ba92bca9d017b49936c55e4)
Features: mac+xpc
Platform: arm64-apple-darwin23.6.0
HighCommander4 commented 1 month ago

@sandymartel based on the stack trace, your crash is in the boost-use-ranges clang-tidy check.

A workaround is to disable the check for clangd by adding the following to .clangd:

Diagnostics:
  ClangTidy:
    Remove: boost-use-ranges

If you can reproduce the crash with clang-tidy on the command line, please file an issue with the clang-tidy tag.

sylvestre commented 1 month ago

Is there any way to make the symbolizer useful for the version in the apt repository?

cc @sylvestre, as point of contact on the apt.llvm.org page, for this question.

probably, could you please open a new issue for this? thanks

bgemmill commented 3 weeks ago

Trying again today with clangd: Ubuntu clangd version 19.1.1 (++20241001124244+d401987fe349-1~exp1~20241001124257.48)

I see a different error for the crashes:

I[18:32:14.044] <-- textDocument/codeAction(28)
I[18:32:14.045] --> reply:textDocument/codeAction(28) 0 ms
I[18:32:14.045] --> textDocument/clangd.fileStatus
I[18:32:14.092] <-- textDocument/documentSymbol(29)
terminate called after throwing an instance of 'std::bad_array_new_length'
  what():  std::bad_array_new_length
[Error - 6:32:18 PM] The Clang Language Server server crashed 5 times in the last 3 minutes. The server will not be restarted. See the output for more information.
[Error - 6:32:18 PM] Request textDocument/documentSymbol failed.
[object Object]
[Error - 6:32:18 PM] Request textDocument/inlayHint failed.
[object Object]
[Error - 6:32:18 PM] Request textDocument/semanticTokens/full/delta failed.
[object Object]
[Error - 6:32:18 PM] Request textDocument/inlayHint failed.
[object Object]

Is this the same thing or a new issue?

HighCommander4 commented 2 weeks ago

Is this the same thing or a new issue?

Does the crash occur with the boost-use-ranges check disabled as shown in this comment?

HighCommander4 commented 2 weeks ago

There's a reduced testcase for the boost-use-ranges crash in #109367.

HighCommander4 commented 2 weeks ago

I'll dupe this over to https://github.com/llvm/llvm-project/issues/109367 which now has a fix posted.

HighCommander4 commented 2 weeks ago

Duplicate of https://github.com/llvm/llvm-project/issues/109367