Open TravellingSalesPerson opened 7 years ago
Thanks for the excellent bug report! Hopefully we can get this resolved for you promptly. To handle a few of the points you brought up, in order:
I figured out what was my problem, i had both lua 5.2 and 5.3 libs in my system, so vim was built with 5.2, but color coded was building with 5.3 for some reason, maybe cmake finds just newest one. Building it with lua5.2 fixed the problem.
We've had a heck of a time trying to reproduce #154 and have not yet succeeded in doing so. It's certainly possible that these are connected, but that's not my current suspicion. I'm gonna try to focus on the SIGSEV on startup first, since that's blocking you from even using color_coded.
Your crash is in color_coded::conf::find
and boost::filesystem::detail::directory_iterator_construct
, which is pretty indicative of what's going on. To me, this screams about filesystem permissions, fuse, and other fs-related issues. Are you in a remote filesystem? Do you have read access to all parents above you up to /
? Are any of your parents symbolic links? What if you run color_coded in a different directory which is closer to /
, more local, with lighter permissions (just to test)?
I think the questions in the third point will lead us to what's going on with your startup crash. If you'd like, you can also add some logs to color_coded::conf::find
, to see where it's going before it crashes.
Thank you for your prompt answer and your enthusiasm! I would love to use color_coded again soon!
apt list --installed | grep lua
liblua5.2-0/zesty,now 5.2.4-1.1build1 amd64 [installed,automatic] liblua5.2-dev/zesty,now 5.2.4-1.1build1 amd64 [installed] lua5.2/zesty,now 5.2.4-1.1build1 amd64 [installed]
Also, I usually remove color_coded by using vundle's `:PluginClean` while making sure that the folder `~/.vim/bundle/color_coded` is deleted. I believe this is enough to uninstall it completely? I assume it is not possible that when reinstalling color_coded it somehow still thinks that there exists lua5.3 because this was the case at the very first install? (Just so that we have covered also this unlikely case)
2. Exactly, I can't even use it at all... :(
3.
a) Not a remote filesystem
b) Yes, read access all the way:
drwxr-xr-x home drwxr-xr-x tsp drwxr-xr-x .vim drwxr-xr-x bundle drwxr-xr-x color_coded
c) None of `home, tsp, .vim, bundle, color_coded` are symbolic links. (~/.vimrc is a symbolic link though)
d) I am not sure how I would do that. If I would `mv color_coded ~` then vi would no longer run it. Could you elaborate?
4.
I didn't do too well here. I tried `std::cout` but nothing showed up in the console. I tried building in debug mode and use gdb, but I was unable to see the source. Then I modified `include/conf/find.hpp` as follows:
/* From the current directory, search for a .color_coded file for the
is found, move to the parent directory. Repeat until /. */ inline std::string find(fs::path file, std::string const &filetype) { auto log = [] (auto arg) { std::ofstream logfile("log.txt", std::ios::app | std::ios::out); logfile << arg << std::endl; logfile.close(); };
auto log2 = [] (auto arg, auto arg2) { //sorry for not making it variadic std::ofstream logfile("log.txt", std::ios::app | std::ios::out); logfile << arg << arg2 << std::endl; logfile.close(); };
log2("arg file", file); log2("arg filetype: ", filetype);
fs::path curr{ file.parent_path() };
log2("curr: ", curr);
static fs::directory_iterator const end;
static auto constexpr file_name(".color_coded"); log2("file_name: ", file_name);
static auto constexpr compilation_database("compile_commands.json");
auto const typed_file_name(std::string{ filename } + "" + filetype);
std::string const flag_files[] {typed_file_name, compilation_database, file_name}; for (int i = 0; i < 3; ++i) { log2("flag_files["+std::to_string(i)+"]: ",flag_files[i]); }
while(true) { for(auto const &file : flag_files) { log2("print loop variable file: ", file); auto const dir_it(find_file(curr, file)); if(dir_it != end) { log("end1"); return dir_it->path().string(); } } log("post for loop"); / We may have hit /. / if(!fs::canonical(curr).has_parent_path()) { log("end2"); return {}; }
curr /= "..";
log2("curr in for loop: ", curr);
} log("end3"); return {}; }
Too bad it's not color_coded, I hope it still helps... I need to close the file all the time because otherwise it usually does not get flushed before the sigsegv.
Anyway, the log.txt file reads after running vi asdf.cpp
arg file"/home/tsp/Documents/asdf.cpp"
arg filetype: cpp
curr: ""
file_name: .color_coded
flag_files[0]: .color_coded_cpp
flag_files[1]: compile_commands.json
flag_files[2]: .color_coded
print loop variable file: .color_coded_cpp
The reason that no "end1/end2/end3" shows up is because it crashes before exiting. Since "post for loop" does not show up either and only one loop iterations shows probably means it dies in the first iteration. But unfortunately, no guarantees because who knows which lines make it to the file and which don't. Because sometimes the output is also simply
arg file"/home/tsp/Documents/qwerqwer.cpp"
arg filetype: cpp
curr:
Does this help? I am very confused and it's getting very late at night. I'll try harder on a new day. Maybe you have a more reliable method of logging in mind if this does not help at all?
I have a more details on this issue:
I have just installed a completely new Ubuntu 17.04 (downloaded from the official website onto a stick).
Besides from setting my username, my region, my keyboard layout and alike, all I did was opening a terminal and using the following commands (copied from .bash_history without modification except misspelled commands):
//obvious stuff:
sudo apt update
sudo apt upgrade
sudo apt install git
git clone https://github.com/VundleVim/Vundle.vim.git ~/.vim/bundle/Vundle.vim
//here some unnecessary openings of vi until I realized i don't have the right version:
vi ~/.vimrc //write minimal .vimrc (see below)
vi //weird vimrc parse error
vi ~/.vimrc //weird vimrc parse error
vi ~/.vimrc //weird vimrc parse error
vim --version //ohhhhhh
sudo apt install vim-gnome //time to install the actual version
vim --version //yesss
vim //no weird vimrc parse error :)
and then I just follow your steps:
sudo apt install build-essential libclang-3.9-dev libncurses-dev libz-dev cmake xz-utils libpthread-workqueue-dev
vim --version
vim --version | grep lua //ok - lua 5.2
sudo apt install liblua5.2-dev lua5.2
gcc --version //6.3.0 - great
vi ~/.vimrc //run :PluginInstall from Vundle to install color_coded
cd .vim/bundle/
cd color_coded/
mkdir build
cd build
cmake ..
make && make install
vi asdf.cpp //same SIGSEGV crash as in previous post when opening the file -> color_coded completely unusable :(
vi asdf //same SIGSEGV crash as in previous post when exiting the file
And finally, here is the .vimrc:
set nocompatible " be iMproved, required
filetype off " required
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()
Plugin 'VundleVim/Vundle.vim'
Plugin 'jeaye/color_coded'
call vundle#end() " required
filetype plugin indent on " required
If you have time, is there anything more of an information I can give to you?
I am not sure what to do. On my main machine and on my laptop there is the same problem. A third machine, with its first couple command only installing vim and color_coded, also has the same issue.
Is it maybe Ubuntu 17.04? Should I try a different version? Is something wrong with the installation steps I do? I am completely happy to reinstall the system again with some other instructions if this helps.
Thanks for the super detailed followup, @TravellingSalesPerson. It's a bummer this is still a problem for you and I will spin up a 17.04 machine so I can debug further.
What you've done and reported has been super helpful and I certainly wouldn't ask you to reinstall your system again. :) I'm aiming to have this resolved over the weekend.
You are actually lucky enough that I wanted to newly install that machine anyway, so do not worry.
And you are even luckier, because I will do a fresh install on my laptop as well in the next couple days. So if you have any wishes, let me know. It's not a big effort to execute a couple lines for installing color_coded first again and I am happy to do that.
Following your steps, with an Ubuntu 17.04 vagrant machine, I've been able to reproduce this crash. However, Ubuntu 17.04 provides multiple vim options and I've found that not all of them have this crash.
After selecting vim-gtk3 again, and running this through gdb (add -O0 -ggdb
to compile flags), I can see what's going on, but I don't yet know why. The related source is here: https://github.com/jeaye/color_coded/blob/master/include/conf/find.hpp#L30
When I run it locally, and break at that line, I see:
(gdb) p file
$1 = {static preferred_separator = 47 '/', m_pathname = "/home/jeaye/.vim/plugged/color_coded/src/main.cpp"}
(gdb) p curr
$2 = {static preferred_separator = 47 '/', m_pathname = "/home/jeaye/.vim/plugged/color_coded/src"}
However, the Ubuntu machine gets something which makes a lot less sense:
(gdb) p file
$1 = {static preferred_separator = 47 '/', m_pathname = "/home/vagrant/.vim/bundle/color_coded/src/main.cpp"}
(gdb) p curr
$2 = {static preferred_separator = 47 '/', m_pathname = ""}
So the parent_path
of the source file, on Ubuntu (with these specific vim builds), ends up being empty. On my Arch box, and on most typical machines, it's the actual parent directory.
Naturally, the vim builds which don't crash also end up with sane output when I connect with gdb.
(gdb) p file
$1 = {static preferred_separator = 47 '/', m_pathname = "/home/vagrant/.vim/bundle/color_coded/src/main.cpp"}
(gdb) p curr
$2 = {static preferred_separator = 47 '/', m_pathname = "/home/vagrant/.vim/bundle/color_coded/src"}
So, what's different? I don't know yet. The crash and weird logic is in boost, but the boost version isn't changing, since color_coded keeps its own boost to help avoid weird situations like this. Maybe the different vim versions are initializing things in different orders, which prevents some statics in boost from being initialized. Needs more investigation.
Use one of the working vims in the list above. Hopefully we can get this crash sorted out promptly; any further research or info would help!
Ok, I believe I see the source of the problem:
# With vim-gtk3 (crashes)
vagrant@vagrant:~/.vim/bundle/color_coded$ ldd $(which vim) | grep boost
libboost_system.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.62.0 (0x00007f255f997000)
libboost_filesystem.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.62.0 (0x00007f255e489000)
vagrant@vagrant:~/.vim/bundle/color_coded$
# With vim-gtk (works)
vagrant@vagrant:~/.vim/bundle/color_coded$ ldd $(which vim) | grep boost
vagrant@vagrant:~/.vim/bundle/color_coded$
These broken vims have their own boost dynamically linked in, which is a different version than the boost color_coded is statically linking. It seems like we're jumping back and forth between them, using the headers that color_coded has, but the dynamic library that vim has linked to.
Even if color_coded were to dynamically link its boost parts, we still have the issue of double linkage here. To support these vims with boost, I think the only thing we can do is make sure we compile specifically against the version they're using and also make sure to link against that one as well. In other words, dynamically linking the system boost seems to be the only solution here (since the duplication will be resolved with both of them just referencing the same shared object).
Very interesting find!
Here also the output for vim-gnome (which crashes) for completeness:
ldd $(which vim) | grep boost
libboost_system.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.62.0 (0x00007f897d3ef000)
libboost_filesystem.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.62.0 (0x00007f897bee1000)
So, the solution is to no longer link to the boost color_coded comes with if the installed vim comes with one (and to link to the same as vim does instead)? Does this simply boil down to a small fix in the cmake files of color_coded?
Unfortunately, it doesn't seem that simple. I've tried changing CMake to build with the dynamic system boost and I end up getting the same crash.
vagrant@vagrant:~/.vim/bundle/color_coded/build$ ldd ../color_coded.so | grep boost
libboost_filesystem.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.62.0 (0x00007f0d773a4000)
libboost_system.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.62.0 (0x00007f0d7719e000)
vagrant@vagrant:~/.vim/bundle/color_coded/build$ ldd $(which vim) | grep boost
libboost_system.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.62.0 (0x00007f5410eea000)
libboost_filesystem.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.62.0 (0x00007f540f9dc000)
My hypothesis could be incorrect, or not correct enough. If you'd like to tinker, I've pushed my hacks for building with the system boost to the system-boost
branch. You should just be able to check it out and re-configure again to get everything. The readme was updated with the new required deps for using the system boost.
Currently out of ideas. I'll take a break from it, for the rest of the day, and see if something comes to mind.
I have tried your system-boost
branch and, as you probably expected, the same crash happens when opening a c/cpp/h file.
However, there is something else, much more interesting: Your system-boost
branch fixes issue #154 for me. That is, if I open anything other than c/cpp/h files, (for example vi asdf.txt
) then I do not get an invalid free when I exit. (Most likely, c/cpp/h files would also close successfully if they would open in the first place)
I switched between master and system-boost multiple times and this result was always reproducible. So your fix seems to have worked, just for the other issue (#154) and not this one. :)
I'd expect that you should be able to observe the same result in your test environment. If you do, then maybe the other issue (#154) is boost related but not this one?
I remember having tried color_coded in the beginning of this year and it always crashed when opening vim (I don't remember if it was only for c/cpp/h files). I did get it fixed. It's a bummer that I do not remember how. I do remember for sure that I looked a lot into #119 because it was also a SIGSEGV and so I played around with installing and removing lua packages. However, I am not sure whether this is what fixed it in the end. Unfortunately, half a year later - now - I got the crash again when installing color_coded to a different machine. I retried what I remembered from the earlier experience but it did not work (which is what lead me here). One difference from back then to now is that used Ubuntu 16.04 or 16.10 and now 17.04. I have no idea if this anecdote of mine is of any help to you, but I thought I might share.
For curiousness, I have tried a different Ubuntu version, because I remember I got color_coded working in the past. I now tried Ubuntu 16.04.3 LTS
I installed it newly and besides from setting my username, my region, my keyboard layout and alike, all I did was opening a terminal and using the following commands (copied from .bash_history without modification):
sudo apt update
sudo apt upgrade
sudo apt install git
git clone https://github.com/VundleVim/Vundle.vim.git ~/.vim/bundle/Vundle.vim
sudo apt install vim-gnome
vi ~/.vimrc //write minimal .vimrc
vi //run :PluginInstall
sudo apt install build-essential libclang-3.9-dev libncurses-dev libz-dev cmake xz-utils libpthread-workqueue-dev
gcc --version //5.4.0 -> this is different than on Ubuntu 16.04 (see above when I did the same with Ubuntu 17.04)
vim --version
vim --version | grep lua //lua 5.2
apt list --installed | grep lua
sudo apt install liblua5.2-dev lua5.2
cd .vim/bundle/color_coded/
mkdir build
cd build
cmake ..
make && make install
vi asdf.cpp //no crash on startup!!
vi //no crash on exit!!
exit
With the .vimrc:
set nocompatible " be iMproved, required
filetype off " required
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()
Plugin 'VundleVim/Vundle.vim'
Plugin 'jeaye/color_coded'
call vundle#end() " required
filetype plugin indent on " required
Therefore, color_coded does not crash when opening c/cpp/h files and neither does it crash when exiting files (issue #154) in Ubuntu 16.04.
So, another workaround (besides using different versions of vim as you pointed out) is using a different Ubuntu version.
I am sure you are curious about the output of
ldd $(which vim) | grep boost
on said system. The output is empty.
In summary: Ubuntu 17.04: -Vim-gnome with color_coded installed crashes when opening c/cpp/h files and when closing any file -Vim-gnome is dynamically linked to the boost libraries
libboost_system.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.62.0 (0x00007f897d3ef000)
libboost_filesystem.so.1.62.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.62.0 (0x00007f897bee1000)
-Working vim versions (e.g. vim-gtk) are not dynamically linked to boost (as ldd $(which vim) | grep boost
is empy)
Ubuntu 16.04
-Vim-gnome with color_coded installed neither crashed when opening c/cpp/h files nor when closing files
-Vim-gnome is not dynamically linked to boost (as ldd $(which vim) | grep boost
is empy)
Therefore, the issue should definitely be boost related. Vim-gnome on 16.04 works and does not link to boost while vim-gnome on 17.04 does not work while linking to boost.
Basically, if future 17.10 vim-gnome reverts to no longer link to boost, the issue is resolved for me by switching to Ubuntu 17.10 and marking vim-gnome on 17.04 as incompatible with color_coded.
Also, recall from my previous post that your system-boost
branch fixes #154 for me. I believe you might be really close to resolving this issue as to me it definitely seems to be boost linking related.
On Ubuntu 20.04, I had the same problem above.
I tried many combination of vim
and lua
.
Eventually, I found the correct combination which makes it work well; vim-nox
, liblua5.2-dev
and lua5.2
Unfortunately, I don't know the reason why it doesn't work well on the others.
No matter whether opening a new file or an existing file, each of the following commands
result in:
Vi only shows up for a fraction of a second.
backtrace from GDB:
My vim version:
Very importantly, this problem does not happen when removing color_coded.
I am running a fresh install of Ubuntu 17.04 with the following vimrc
I have the same issue on my 2nd machine (also Ubuntu 17.04).
The symptoms are the same as in https://github.com/jeaye/color_coded/issues/119, but the stack trace is different and clang is 3.9.
I believe this issue could be related to https://github.com/jeaye/color_coded/issues/154, because I also get an invalid free whenever I close vim (except with c, cpp, h files because I don't even get to open them)
In case the two issues are indeed related, here is the output of valgrind when closing vim:
Where one line is color_coded related:
==19221== by 0x15096913: _GLOBAL__sub_I_path.cpp (in /home/tsp/.vim/bundle/color_coded/color_coded.so)