Closed sean-mcmanus closed 1 year ago
sean-mcmanus
commented
2 years ago @lucasaf04 Your issue appears different. Our main thread is stuck calling the wordexp system call. Are you able to provide the Custom Browse Configuration logging in the C/C++ logging window after setting C_Cpp.loggingLevel to "Debug", in particular the compilerFragments section?
lucasaf04
commented
2 years ago @sean-mcmanus
Colengms
commented
2 years ago Hi @lucasaf04 . It looks like our call to the OS api function wordexp is simply not returning. I'm not able easily to reproduce the issue on my mac with the same compilerFragments. The call to wordexp is used to resolve compilerFragments in the same way that the current shell would, such as to remove shell quoting and escaping. https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/wordexp.3.html
The compilerFragments in your logs are not unusual. They don't even contain shell quoting or escaping. And we're not yet getting reports about this issue from other Mac users. Is there anything unusual about how the shell is configured on your system?
sean-mcmanus
commented
2 years ago Yeah, same here.
lucasaf04
commented
2 years ago @Colengms Here is my .zshenv, if .zshrc is also necessary I'll edit this message.
lucasaf04
commented
2 years ago @Colengms Any news?
sean-mcmanus
commented
2 years ago Our team isn't able to repro the issue.
@Colengms Has anyone tried the repro with the .zshenv? I did not.
Colengms
commented
2 years ago @v-ericawu When you tried to repro this, were you able to set up an environment that leveraged the provided .zshenv file?
v-ericawu
commented
2 years ago @Colengms IntelliSense loads normally after setting the .zshenv file:
lucasaf04
commented
2 years ago @v-ericawu It also does for me. The problem occurs somewhere in between editing and running the target. Sometimes it happens on the first run, sometimes after several runs. That's why I couldn't make a video reproing the problem, it happens randomly. I don't know what else I can do to help. Hope it gets fixed soon because its very annoying having to go to activity monitor to kill all cpptools processes to fix IntelliSense (restarting vscode without manual kill of cpptools keeps the processes alive)
lucasaf04
commented
2 years ago @Colengms I have a video reproing the issue (most of it is just editing code and running the program) Hope it helps.
albertcaldas84
commented
2 years ago Anyone knows any workaround for these hangs? I'm having them so often (every minute or so) that this is hintering development completely. Killing the cpptools-srv doesn't work anymore. Most of the times I've looked into it, it's the same call to wordexp. Can I help with more callstacks or something? I've just enabled logging again. I think that it happens only after compiling, but will try to double check.
lucasaf04
commented
2 years ago Anyone knows any workaround for these hangs? I'm having them so often (every minute or so) that this is hintering development completely. Killing the cpptools-srv doesn't work anymore. Most of the times I've looked into it, it's the same call to wordexp. Can I help with more callstacks or something? I've just enabled logging again. I think that it happens only after compiling, but will try to double check.
Its getting really annoying for me as well. Killing cpptools and cpptools-srv fixes the error, but only until the next hang. If there's something we could do to help fix it asap.
sean-mcmanus
commented
2 years ago What Mac OS version is everyone using who is hitting this?
It seems like a bug with the Mac OS, since it's stuck inside their implementation of wordexp, and in their documentation I don't see any information on why wordexp would get stuck: https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/wordexp.3.html . Can you file a bug on your OS? https://developer.apple.com/bug-reporting/ We haven't gotten any bug reports from Linux users.
Maybe we could add some workaround for this Mac bug via running it in a separate thread and just continue processing after a timeout (although the thread would still be stuck/leaked).
lucasaf04
commented
2 years ago @sean-mcmanus I can't barely remember the first time I hit the issue, I think Big Sur didn't have this problem, so it seems like a Monterey one. The only thing I'm sure is that 12.4, 12.5 and 12.6 have it. (now I'm on 12.6)
In relation to the workaround, would be possible to detect the stuck thread and kill it?
lucasaf04
commented
2 years ago @sean-mcmanus Man page for wordexp in my system is different than yours: (see implementation, diagnostics and bugs sections)
~
❯ man --path wordexp
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/share/man/man3/wordexp.3
~
❯ man wordexp | col -b > wordexp_man_page_12.6_.txt
Note: MacOSX.sdk is a sym link to MacOSX12.3.sdk
sean-mcmanus
commented
2 years ago Are you able to build/run this sample program that uses wordexp, i.e. are you able to make it get stuck somehow via passing in the arguments that we're using in your compilerFragments (in your logging). It works for me and doesn't get stuck (using macOS 12.4), i.e. it keeps logging 0.
#include <wordexp.h>
#include <iostream>
int main()
{
wordexp_t worde;
const char numargs = 24;
const char *strs[numargs] = {
"-g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk",
"-fcolor-diagnostics",
"-I/opt/homebrew/opt/llvm/include",
"-I'/opt/homebrew/opt/llvm/include'",
"-I\"/opt/homebrew/opt/llvm/include\"",
"-I\\\\\"/opt/homebrew/opt/llvm/include\\\\\"",
"-Wall",
"-Weffc++",
"-Wextra",
"-Wpedantic",
"-Wshadow",
"-Wunused",
"-Wsign-conversion",
"-Wnon-virtual-dtor",
"-Wold-style-cast",
"-Wcast-align",
"-Woverloaded-virtual",
"-Wconversion",
"-Wnull-dereference",
"-Wdouble-promotion",
"-Wformat=2",
"-Wimplicit-fallthrough",
"-Werror",
"-std=c++11"};
int argn = 0;
while (true)
{
int i = wordexp(strs[argn++ % numargs], &worde, 0); // maybe replace 0 with WRDE_NOCMD?
wordfree(&worde);
if (i != 0)
break;
std::cout << i << std::endl;
}
return 0;
}The docs say "The wordexp() function attempts to detect input that would cause commands to be execute", so if it's getting stuck on that code, we can try passing in WRDE_NOCMD as a potential workaround.
FYI, I found a bug with wordfree wasn't being call on error conditions, but I'm not seeing that cause it to get stuck.
Yeah, maybe we could cancel the thread, but our threading API doesn't currently support it...might be easy to add though, i.e. pthread_cancel.
Also, if the bug doesn't repro with a modified or non-existant .zshenv, it could be some issue with that, since it looks like wordexp executes the shell internally, which could execute the .zshenv, so maybe it's getting stuck on that for some reason.
lucasaf04
commented
2 years ago @sean-mcmanus No issues running this code. IDK why cpptools sometimes gets stuck. Maybe you could upload a debug build of the extension that reports every call to wordexp and its arguments to see where it gets stuck, and another one with WRDE_NOCMD to see if it fixes the error. I'll continue playing with the code to see if I get wordexp stuck. When you say stuck you mean non zero return from wordexp or no return at all?
lucasaf04
commented
2 years ago Research: If you force some error on the first call to wordexp and then you call wordfree(&worde) you get this:
playground(50963,0x104dd0580) malloc: *** error for object 0x1: pointer being freed was not allocated
playground(50963,0x104dd0580) malloc: *** set a breakpoint in malloc_error_break to debug
[1] 50963 abortcall stack of sample program while running correctly:
❯ lldb -p 69599
(lldb) process attach --pid 69599
Process 69599 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
frame #0: 0x00000001861ac9ec libsystem_kernel.dylib`__read_nocancel + 8
libsystem_kernel.dylib`:
-> 0x1861ac9ec <+8>: b.lo 0x1861aca0c ; <+40>
0x1861ac9f0 <+12>: pacibsp
0x1861ac9f4 <+16>: stp x29, x30, [sp, #-0x10]!
0x1861ac9f8 <+20>: mov x29, sp
Executable module set to "/Users/lucasaf04/Developer/Tutorials-and-Learning/Cpp11/Cpp-Primer/build/src/playground".
Architecture set to: arm64e-apple-macosx-.
(lldb) bt all
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
* frame #0: 0x00000001861ac9ec libsystem_kernel.dylib`__read_nocancel + 8
frame #1: 0x0000000186112ab4 libsystem_c.dylib`we_read_fully + 52
frame #2: 0x0000000186112708 libsystem_c.dylib`wordexp + 1668
frame #3: 0x000000010497eebc playground`main at playground.cpp:47:17
frame #4: 0x0000000104c1508c dyld`start + 520
(lldb) exit
Quitting LLDB will detach from one or more processes. Do you really want to proceed: [Y/n] y
Note: I got intellisense stuck a couple times while editing the sample code.
sean-mcmanus
commented
2 years ago Yes, we're looking into potentially adding Mac-only input logging and disabling commands for 1.13.1 (or a special debug vsix?). By stuck, I mean wordexp isn't returning (unless you think it's being called repeatedly in some loop, i.e. you'd probably see a lot of CPU usage).
Yeah, we decided against calling wordfree on errors, since it looks like the API only intends wordfree to be called on non-error conditions.
lucasaf04
commented
1 year ago @sean-mcmanus Looking forward 1.13.1 release.
This are cpptools stats when stuck:
Here is the sample taken with activity monitor: Sample of cpptools.txt
sean-mcmanus
commented
1 year ago Okay, it appears stuck on the single call to wordexp.
albertcaldas84
commented
1 year ago Any news on this?
sean-mcmanus
commented
1 year ago @albertcaldas84 Yeah, we have some changes related to this for our next 1.13.1 update (it was delayed this past week).
sean-mcmanus
commented
1 year ago We've published a version with wordexp-related updates (Mac-only): 1.13.1: https://github.com/microsoft/vscode-cpptools/releases/tag/v1.13.1 -- we made some minor changes that are always enabled, but I'm guessing those won't fix the issue, so to enable the other changes we added you need to set C_Cpp.experimentalFeatures to "enabled" (no other experimental features will get enabled currently) -- that will disable command expansion for wordexp in case that is the cause. If the issue is not fixed with that, then set C_Cpp.loggingLevel to hidden value of "7" (in the settings json editor) to cause "wordexp input: " logging to appear in the C/C++ log which could help identify if there are particular inputs to wordexp that are causing the issue, and those inputs could potentially be used for getting a repro of the bug.
UPDATE: And the loggingLevel should be changed back to "Debug" or some other value after the wordexp input logging is obtained or the heavy logging might cause performance slow down.
albertcaldas84
commented
1 year ago Awesome! Trying it with experimental feature on.
albertcaldas84
commented
1 year ago Still hanging:
thread #15
frame #0: 0x00000001a1b0aa0c libsystem_kernel.dylib`__read_nocancel + 8
frame #1: 0x00000001a1a70930 libsystem_c.dylib`wordexp + 2220
frame #2: 0x00000001058453f0 cpptools`msvc::parse_arguments(msvc::basic_zstring_view<char>, bool) + 108
frame #3: 0x0000000104f5c794 cpptools`compiler_info::parse_arguments(msvc::basic_zstring_view<char>) + 160
frame #4: 0x0000000104f7dc5c cpptools`cpp_properties::set_custom_browse_path(vscode::WorkspaceBrowseConfiguration const&) + 764
frame #5: 0x0000000104feab9c cpptools`vscode::message_handler::cpptools_didChangeCustomBrowseConfiguration(vscode::CustomBrowseConfigurationParams) + 124
frame #6: 0x0000000104fd7cf8 cpptools`vscode::message_handler::dispatch(vscode::vscode_client_message&&, vscode::vscode_server_message&, vscode::message_handler::msg_proc_thread_token) + 10308
frame #7: 0x0000000104fd53e8 cpptools`vscode::message_handler::handle_message(vscode::vscode_client_message&&, vscode::message_handler::msg_proc_thread_token) + 52
frame #8: 0x000000010504445c cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_7, std::__1::allocator<vscode::message_handler::main_loop()::$_7>, void ()>::operator()() + 1996
frame #9: 0x0000000105844f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #10: 0x00000001a1b4626c libsystem_pthread.dylib`_pthread_start + 148will enable logging.
albertcaldas84
commented
1 year ago sean-mcmanus
commented
1 year ago Logging indicates that the call to wordexp("-std=c++17") is getting stuck. All the other calls to wordexp seem fine too. So there doesn't appear to be anything obviously wrong going on. Maybe we just need to add a timeout/cancel.
sean-mcmanus
commented
1 year ago The fix for https://github.com/microsoft/vscode-cpptools/issues/9882 may have a side effect of fixing (or reducing the occurrence of) this issue by not repeatedly calling wordexp on the same inputs.
sean-mcmanus
commented
1 year ago Your logging shows it get stuck on the 18th call to wordexp -- is the call that gets stuck random? i.e. sometimes is sooner or later? If so, that would indicate there is some race condition, possibly due to the other IntelliSense thread doing process creation work that could interfere with wordexp's internal process creation if they happen to occur at the same time.
albertcaldas84
commented
1 year ago Often it gets stuck in 5 minutes, but sometimes in 20. Seems pretty random.
lucasaf04
commented
1 year ago Intellisense got stuck using experimental and debug set to 7:
~
❯ lldb -p 16942
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/homebrew/Cellar/llvm/15.0.1/libexec/python3.10/site-packages/lldb/__init__.py", line 99, in <module>
import six
ModuleNotFoundError: No module named 'six'
(lldb) process attach --pid 16942
Process 16942 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
frame #0: 0x00000001b71fc9ec libsystem_kernel.dylib`__read_nocancel + 8
libsystem_kernel.dylib`:
-> 0x1b71fc9ec <+8>: b.lo 0x1b71fca0c ; <+40>
0x1b71fc9f0 <+12>: pacibsp
0x1b71fc9f4 <+16>: stp x29, x30, [sp, #-0x10]!
0x1b71fc9f8 <+20>: mov x29, sp
Executable module set to "/Users/lucasaf04/.vscode/extensions/ms-vscode.cpptools-1.13.1-darwin-arm64/bin/cpptools".
Architecture set to: arm64e-apple-macosx-.
(lldb) bt all
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
* frame #0: 0x00000001b71fc9ec libsystem_kernel.dylib`__read_nocancel + 8
frame #1: 0x00000001b712a714 libsystem_c.dylib`__sread + 24
frame #2: 0x00000001b7105618 libsystem_c.dylib`_sread + 32
frame #3: 0x00000001b71055ac libsystem_c.dylib`__srefill1 + 36
frame #4: 0x00000001b7120a78 libsystem_c.dylib`__srget + 24
frame #5: 0x00000001b712eba0 libsystem_c.dylib`getc + 72
frame #6: 0x00000001b71a781c libc++.1.dylib`std::__1::__stdinbuf<char>::__getchar(bool) + 160
frame #7: 0x00000001002ee360 cpptools`std::__1::basic_istream<char, std::__1::char_traits<char>>& std::__1::getline<char, std::__1::char_traits<char>, std::__1::allocator<char>>(std::__1::basic_istream<char, std::__1::char_traits<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, char) + 152
frame #8: 0x00000001002af1a4 cpptools`vscode::message_handler::main_loop() + 1644
frame #9: 0x00000001002ad990 cpptools`main + 288
frame #10: 0x0000000101bd108c dyld`start + 520
thread #2
frame #0: 0x00000001b7200c20 libsystem_kernel.dylib`kevent + 8
frame #1: 0x0000000100b3a630 cpptools`uv__io_poll + 724
frame #2: 0x0000000100b33c8c cpptools`uv_run + 392
frame #3: 0x0000000100af95a0 cpptools`msvc::loop_t::run_loop() + 64
frame #4: 0x0000000100afc640 cpptools`msvc::thread_t::invoker_t<void (*)(std::__1::shared_ptr<msvc::loop_t>), std::__1::shared_ptr<msvc::loop_t>&>::invoke() + 40
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #3
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003af730 cpptools`vscode::thread_pool::do_work(unsigned long) + 340
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #4
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003af730 cpptools`vscode::thread_pool::do_work(unsigned long) + 340
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #5
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003af730 cpptools`vscode::thread_pool::do_work(unsigned long) + 340
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #6
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003af730 cpptools`vscode::thread_pool::do_work(unsigned long) + 340
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #7
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003af730 cpptools`vscode::thread_pool::do_work(unsigned long) + 340
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #8
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003af730 cpptools`vscode::thread_pool::do_work(unsigned long) + 340
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #9
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003af730 cpptools`vscode::thread_pool::do_work(unsigned long) + 340
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #10
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003af730 cpptools`vscode::thread_pool::do_work(unsigned long) + 340
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #11
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187320 libc++.1.dylib`std::__1::condition_variable::__do_timed_wait(std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l>>>) + 100
frame #3: 0x0000000100296804 cpptools`std::__1::cv_status std::__1::condition_variable_any::wait_until<std::__1::unique_lock<std::__1::mutex>, std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l>>>(std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l>>> const&) + 268
frame #4: 0x00000001003b100c cpptools`unsigned int msvc::bitset_event_t::wait_for_any_set<long long, std::__1::ratio<1l, 1000l>>(std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>> const&, unsigned int) + 112
frame #5: 0x00000001003b0ee0 cpptools`msvc::thread_t::invoker_t<vscode::thread_pool::thread_pool()::$_0>::invoke() + 164
frame #6: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #7: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #12
frame #0: 0x00000001b7200c20 libsystem_kernel.dylib`kevent + 8
frame #1: 0x0000000100b3a630 cpptools`uv__io_poll + 724
frame #2: 0x0000000100b33c8c cpptools`uv_run + 392
frame #3: 0x0000000100af95a0 cpptools`msvc::loop_t::run_loop() + 64
frame #4: 0x0000000100afc640 cpptools`msvc::thread_t::invoker_t<void (*)(std::__1::shared_ptr<msvc::loop_t>), std::__1::shared_ptr<msvc::loop_t>&>::invoke() + 40
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #13
frame #0: 0x00000001b71fc9ec libsystem_kernel.dylib`__read_nocancel + 8
frame #1: 0x00000001b7162930 libsystem_c.dylib`wordexp + 2220
frame #2: 0x0000000100b213f0 cpptools`msvc::parse_arguments(msvc::basic_zstring_view<char>, bool) + 108
frame #3: 0x0000000100238794 cpptools`compiler_info::parse_arguments(msvc::basic_zstring_view<char>) + 160
frame #4: 0x0000000100259c5c cpptools`cpp_properties::set_custom_browse_path(vscode::WorkspaceBrowseConfiguration const&) + 764
frame #5: 0x00000001002c6b9c cpptools`vscode::message_handler::cpptools_didChangeCustomBrowseConfiguration(vscode::CustomBrowseConfigurationParams) + 124
frame #6: 0x00000001002b3cf8 cpptools`vscode::message_handler::dispatch(vscode::vscode_client_message&&, vscode::vscode_server_message&, vscode::message_handler::msg_proc_thread_token) + 10308
frame #7: 0x00000001002b13e8 cpptools`vscode::message_handler::handle_message(vscode::vscode_client_message&&, vscode::message_handler::msg_proc_thread_token) + 52
frame #8: 0x000000010032045c cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_7, std::__1::allocator<vscode::message_handler::main_loop()::$_7>, void ()>::operator()() + 1996
frame #9: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #10: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #14
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x00000001003214b0 cpptools`vscode::message_deque<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, false>::pop_impl(bool) + 92
frame #4: 0x0000000100321364 cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_8, std::__1::allocator<vscode::message_handler::main_loop()::$_8>, void ()>::operator()() + 48
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #15
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100322aec cpptools`vscode::message_deque<vscode::folding_ranges_params, false>::pop_impl(bool) + 92
frame #4: 0x0000000100322924 cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_11, std::__1::allocator<vscode::message_handler::main_loop()::$_11>, void ()>::operator()() + 80
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #16
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100321f58 cpptools`vscode::message_deque<vscode::browse_engine_update_action, false>::pop_impl(bool) + 84
frame #4: 0x00000001003217f4 cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_9, std::__1::allocator<vscode::message_handler::main_loop()::$_9>, void ()>::operator()() + 156
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #17
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x0000000100b41260 cpptools`uv_cond_wait + 12
frame #3: 0x0000000100b30674 cpptools`worker + 112
frame #4: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #18
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x0000000100b41260 cpptools`uv_cond_wait + 12
frame #3: 0x0000000100b30674 cpptools`worker + 112
frame #4: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #19
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x0000000100b41260 cpptools`uv_cond_wait + 12
frame #3: 0x0000000100b30674 cpptools`worker + 112
frame #4: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #20
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x0000000100b41260 cpptools`uv_cond_wait + 12
frame #3: 0x0000000100b30674 cpptools`worker + 112
frame #4: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #21
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100323024 cpptools`vscode::message_deque<vscode::vscode_client_message, false>::pop_impl(bool) + 96
frame #4: 0x0000000100322edc cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_12, std::__1::allocator<vscode::message_handler::main_loop()::$_12>, void ()>::operator()() + 68
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #22
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100323024 cpptools`vscode::message_deque<vscode::vscode_client_message, false>::pop_impl(bool) + 96
frame #4: 0x00000001003231dc cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_13, std::__1::allocator<vscode::message_handler::main_loop()::$_13>, void ()>::operator()() + 68
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #23
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x00000001003237a8 cpptools`vscode::message_deque<vscode::message_handler::parse_file_entry, true>::pop_impl(bool) + 92
frame #4: 0x000000010032338c cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_14, std::__1::allocator<vscode::message_handler::main_loop()::$_14>, void ()>::operator()() + 112
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #24
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100323024 cpptools`vscode::message_deque<vscode::vscode_client_message, false>::pop_impl(bool) + 96
frame #4: 0x0000000100323ce4 cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_15, std::__1::allocator<vscode::message_handler::main_loop()::$_15>, void ()>::operator()() + 68
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #25
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187320 libc++.1.dylib`std::__1::condition_variable::__do_timed_wait(std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l>>>) + 100
frame #3: 0x00000001003240a8 cpptools`vscode::message_deque<int, false>::pop_impl(bool, int&, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>) + 148
frame #4: 0x0000000100323f5c cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_16, std::__1::allocator<vscode::message_handler::main_loop()::$_16>, void ()>::operator()() + 316
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #26
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100296450 cpptools`void std::__1::condition_variable_any::wait<std::__1::unique_lock<std::__1::mutex>>(std::__1::unique_lock<std::__1::mutex>&) + 100
frame #4: 0x00000001003243f8 cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_17, std::__1::allocator<vscode::message_handler::main_loop()::$_17>, void ()>::operator()() + 196
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #27
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x0000000100322700 cpptools`vscode::message_deque<int, false>::pop_impl(bool) + 84
frame #4: 0x0000000100322628 cpptools`std::__1::__function::__func<vscode::message_handler::main_loop()::$_10, std::__1::allocator<vscode::message_handler::main_loop()::$_10>, void ()>::operator()() + 52
frame #5: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #6: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #28
frame #0: 0x00000001b71fe270 libsystem_kernel.dylib`__psynch_cvwait + 8
frame #1: 0x00000001b723883c libsystem_pthread.dylib`_pthread_cond_wait + 1236
frame #2: 0x00000001b7187284 libc++.1.dylib`std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
frame #3: 0x000000010093f8b8 cpptools`run_tag_parser() + 348
frame #4: 0x00000001008d3160 cpptools`process_translation_unit(char const*, int, an_exported_template_file*) + 912
frame #5: 0x0000000100548c44 cpptools`cfe_main(int, char**) + 132
frame #6: 0x00000001008f94c4 cpptools`cfe_main_exception_handler(int, char**) + 12
frame #7: 0x0000000100548d30 cpptools`edg_main(int, char**) + 12
frame #8: 0x0000000100943314 cpptools`antlr_parse_routine() + 488
frame #9: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #10: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
thread #29
frame #0: 0x00000001b71fe06c libsystem_kernel.dylib`__semwait_signal + 8
frame #1: 0x00000001b7106fc8 libsystem_c.dylib`nanosleep + 220
frame #2: 0x00000001b7193af8 libc++.1.dylib`std::__1::this_thread::sleep_for(std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l>> const&) + 84
frame #3: 0x00000001003a7ff4 cpptools`vscode::code_analysis::work_scheduler::process_work() + 156
frame #4: 0x0000000100395e28 cpptools`vscode::code_analysis::work_scheduler::main() + 28
frame #5: 0x0000000100395d78 cpptools`vscode::code_analysis::work_scheduler_work(unsigned int) + 92
frame #6: 0x0000000100b20f94 cpptools`msvc::thread_helper_t::thread_entry(void*) + 28
frame #7: 0x00000001b723826c libsystem_pthread.dylib`_pthread_start + 148
(lldb) q
Quitting LLDB will detach from one or more processes. Do you really want to proceed: [Y/n] y
~ took 8s
❯Yeah, we have a change for 1.13.2 that should improve or fix this (our target is next Tuesday).
Zingam
commented
1 year ago I also experience such issue on macOS. Just hovering the mouse over something to see the details can result in the "Loading..." hover and the flame icon in the status bar is stuck on Updating IntelliSense forever.
sean-mcmanus
commented
1 year ago @Zingam Yeah, we believe a significant number of Mac users are randomly hitting this, most of whom probably aren't aware they're hitting this particular bug. I'm not 100% sure you're hitting this or not without knowing the cpptools call stacks, but you should know after we release the fix.
sean-mcmanus
commented
1 year ago https://github.com/microsoft/vscode-cpptools/releases/tag/v1.13.2 has a change that may reduce the occurrence of this but it's not fixed yet. Setting C_Cpp.experimentalFeatures to true may fix this, but we're still running tests to determine if that's true or not. There is a separate "Failed to spawn IntelliSense process" failure that can occur on Mac-only which we're also investigating, but no thread gets stuck in that case.
albertcaldas84
commented
1 year ago Ok, will get it. Thanks! Anything you want us to test?
sean-mcmanus
commented
1 year ago Well, we've found a way to repro the bug(s) and test the potential fix(es) ourselves (still in progress), so we don't necessarily need anyone to test anything...up to you.
albertcaldas84
commented
1 year ago That sounds awesome. I've enabled the experimental feature anyway, will tell.
sean-mcmanus
commented
1 year ago Our testing indicates it's fixed when C_Cpp.experimentalFeatures is set to true, so we'll move those changes to the non-experimental case for 1.13.3.
H-G-Hristov
commented
1 year ago With 1.13.2 I still happened that I was unable to cmd+click to a definition of a Core Foundation type (Apple SDK header). There was no stuck flame.
After I restarted VSCode the issue fixed itself.
Is cmd+click issue related to the wordexp issue?
sean-mcmanus
commented
1 year ago @H-G-Hristov That sounds like a different issue. If wordexp is stuck then all IntelliSense operations would be stuck and wordexp would be on a call stack. If only Go to Definition doesn't work that could be caused by something else. Are you able to provide more repro info or logging or a call stack? It sounds like it's random?
albertcaldas84
commented
1 year ago I confirm that I haven't had any hang in the whole morning. Thanks sean :)
lucasaf04
commented
1 year ago After using the extension for several days I haven't had any issues.
@sean-mcmanus It would be nice to have a tiny explanation of what exactly caused the bug and how you fixed it. Just curiosity, because it has been such an annoying bug.
Thanks.
sean-mcmanus
commented
1 year ago The wordexp call was launching a process internally on Mac and some Linux implementations and that would cause handles to be inherited by that process if another of our threads was creating an IntelliSense process at the same time, so it would be random. Our automated tests managed to repro it in a loop. So we just added a lock to prevent that. We'd hit similar issues in the past.
lucasaf04
commented
1 year ago Thanks :)
H-G-Hristov
commented
1 year ago I notices that the release notes changed from "Fixed and reduced" to "Reduced" is this still not fixed fully? The situation with the preview release is certainly significantly better than it used to be. In a matter of fact I can't remember if the issue has occurred to me since the last update.
sean-mcmanus
commented
1 year ago The fix is available with 1.13.3 (pre-release): https://github.com/microsoft/vscode-cpptools/releases/tag/v1.13.3 -- @H-G-Hristov with 1.13.2 the experimentalFeatures setting had to be set in order for it to be fully fixed (otherwise it was just "reduced").
sean-mcmanus
commented
1 year ago This has been fixed with insiders for over a month, but it's now available in non-insiders: https://github.com/microsoft/vscode-cpptools/releases/tag/v1.13.6
@sean-mcmanus Recently I have been facing the same problem: formatter stuck when saving (I have auto save and format on save active) and intellisense stuck in the flame icon forever (hovering a variable prints
loading...). I don't have a step by step guide to repro the problem because it doesn't happen always. The only thing I do is: edit code and then cmake run (with the cmake-tools ext). Sometimes the bug appears sometimes not.Here is my call stack (I couldn't get it with the vscode - launch.json guide so I used the lldb one):
call stack
``` ~ ❯ lldb -p 79097 (lldb) process attach --pid 79097 Process 79097 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP frame #0: 0x00000001c1e089ec libsystem_kernel.dylib`__read_nocancel + 8 libsystem_kernel.dylib`: -> 0x1c1e089ec <+8>: b.lo 0x1c1e08a0c ; <+40> 0x1c1e089f0 <+12>: pacibsp 0x1c1e089f4 <+16>: stp x29, x30, [sp, #-0x10]! 0x1c1e089f8 <+20>: mov x29, sp Executable module set to "/Users/$USER/.vscode/extensions/ms-vscode.cpptools-1.12.0-darwin-arm64/bin/cpptools". Architecture set to: arm64e-apple-macosx-. (lldb) bt all * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP * frame #0: 0x00000001c1e089ec libsystem_kernel.dylib`__read_nocancel + 8 frame #1: 0x00000001c1d36714 libsystem_c.dylib`__sread + 24 frame #2: 0x00000001c1d11618 libsystem_c.dylib`_sread + 32 frame #3: 0x00000001c1d115ac libsystem_c.dylib`__srefill1 + 36 frame #4: 0x00000001c1d2ca78 libsystem_c.dylib`__srget + 24 frame #5: 0x00000001c1d3aba0 libsystem_c.dylib`getc + 72 frame #6: 0x00000001c1db381c libc++.1.dylib`std::__1::__stdinbufOriginally posted by @lucasaf04 in https://github.com/microsoft/vscode-cpptools/issues/9631#issuecomment-1205032803