Open tdjordan opened 3 years ago
it looks like it is coming from the tree-sitter test
step that is being invoked during the yarn test
portion.
❯ yarn test 20:08:02
yarn run v1.22.11
$ tree-sitter test
Parser compilation failed.
Stdout:
Stderr: /Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:97:39: error: expected expression
return std::vector<TokenType>({ lhs, static_cast<TokenType>(rhs) });
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:100:27: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
std::vector<TokenType>&& operator|(std::vector<TokenType>&& lhs, TokenType rhs)
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:100:62: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
std::vector<TokenType>&& operator|(std::vector<TokenType>&& lhs, TokenType rhs)
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:327:41: error: expected ';' at end of declaration list
std::vector<size_t>& get_tag_stack() noexcept { return m_TagStack; }
^
;
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:556:27: warning: in-class initialization of non-static data member is a C++11 extension [-Wc++11-extensions]
TokenType m_LastToken = NONE;
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:559:26: warning: in-class initialization of non-static data member is a C++11 extension [-Wc++11-extensions]
size_t m_ParsedChars = 0, m_IndentationLevel = 0;
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:559:50: warning: in-class initialization of non-static data member is a C++11 extension [-Wc++11-extensions]
size_t m_ParsedChars = 0, m_IndentationLevel = 0;
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:565:54: warning: in-class initialization of non-static data member is a C++11 extension [-Wc++11-extensions]
const std::array<int32_t, 6> s_DetachedModifiers = { '*', '-', '>', '|', '=', '~' };
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:121:13: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:131:13: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:141:17: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:148:21: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:162:13: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:167:48: error: use of undeclared identifier 'm_Current'
if (lexer->lookahead == ']' && m_Current != '\\')
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:169:21: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:177:17: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:184:20: error: use of undeclared identifier 'check_link'
return check_link(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:188:13: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:206:17: error: use of undeclared identifier 'skip'
skip(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:211:17: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:217:21: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:220:25: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:223:29: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:227:37: error: use of undeclared identifier 'advance'
advance(lexer);
^
/Users/tjordan/.local/share/nvim/tree-sitter-norg-old/src/scanner.cc:252:25: error: use of undeclared identifier 'advance'
advance(lexer);
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
6 warnings and 20 errors generated.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Hmmm, I am getting this on a fresh clone.
❯ node-gyp build 20:30:06
gyp info it worked if it ends with ok
gyp info using node-gyp@8.2.0
gyp info using node@16.8.0 | darwin | x64
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: *** No rule to make target `../../../../../usr/lib/node_modules/node-gyp/addon.gypi', needed by `Makefile'. Stop.
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/Users/tjordan/.config/yarn/global/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack at ChildProcess.emit (node:events:394:28)
gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
gyp ERR! System Darwin 20.6.0
gyp ERR! command "/usr/local/Cellar/node/16.8.0/bin/node" "/usr/local/bin/node-gyp" "build"
gyp ERR! cwd /Users/tjordan/devRoot/treesitter/tree-sitter-norg
gyp ERR! node -v v16.8.0
gyp ERR! node-gyp -v v8.2.0
gyp ERR! not ok
We know as of right now that installation on macos is rather painful. We're also hoping to work with the nvim-treesitter guys to make the process easier. Danymat has a temporary workaround for compiling on mac here, you might wanna check it out.
that link provides a solution that, I think, is a bit too 'dirty'. no need to mess with global cc
, you can override it just for installation like so:
brew install gcc
CC=gcc-11 nvim -c "TSInstall norg"
Yes, this is what I wrote here: https://github.com/nvim-neorg/neorg/issues/74#issuecomment-963428094
Trying to use
CC=gcc-11 nvim
But I've got another error:
nvim-treesitter[norg]: Error during compilation
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/wait.h:110,
from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdlib.h:66,
from /usr/local/Cellar/gcc/11.2.0_3/include/c++/11/cstdlib:75,
from /usr/local/Cellar/gcc/11.2.0_3/include/c++/11/bits/stl_algo.h:59,
from /usr/local/Cellar/gcc/11.2.0_3/include/c++/11/algorithm:62,
from src/scanner.cc:1:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/resource.h:443:34: error: expected initializer before '__OSX_AVAILABLE_STARTING'
443 | int getiopolicy_np(int, int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
| ^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/resource.h:449:39: error: expected initializer before '__OSX_AVAILABLE_STARTING'
449 | int setiopolicy_np(int, int, int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
| ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdlib.h:128,
from /usr/local/Cellar/gcc/11.2.0_3/include/c++/11/cstdlib:75,
from /usr/local/Cellar/gcc/11.2.0_3/include/c++/11/bits/stl_algo.h:59,
from /usr/local/Cellar/gcc/11.2.0_3/include/c++/11/algorithm:62,
from src/scanner.cc:1:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/malloc/_malloc.h:52:77: error: expected initializer before '__OSX_AVAILABLE_STARTING'
...
I'm not sure why, I thought it was because of my SDK(?)
Here's my sdk location
❯ ls -l /Library/Developer/CommandLineTools/SDKs/
total 0
lrwxr-xr-x 1 root wheel 15 Oct 10 2019 MacOSX.sdk -> MacOSX10.15.sdk
drwxr-xr-x 7 root wheel 224 Jul 16 2019 MacOSX10.14.sdk
drwxr-xr-x 8 root wheel 256 Aug 30 2019 MacOSX10.15.sdk
And here's my brew config
HOMEBREW_VERSION: 3.3.10
ORIGIN: https://github.com/Homebrew/brew
HEAD: 385892f3d27dcab0f5a1cb47c3927d9d65ed007c
Last commit: 22 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: c3d7d6ff280e2a907f2e1897d09cb90e77ffb279
Core tap last commit: 57 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_CORE_GIT_REMOTE: https://github.com/Homebrew/homebrew-core
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.8BvWKdiBS3/org.macosforge.xquartz:0
HOMEBREW_MAKE_JOBS: 12
Homebrew Ruby: 2.6.8 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.8/bin/ruby
CPU: dodeca-core 64-bit kabylake
Clang: 12.0.0 build 1200
Git: 2.34.1 => /usr/local/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 10.15.7-x86_64
CLT: 11.0.0.33.8
Xcode: 12.4
And here's my checkhealth nvim_treesitter
nvim_treesitter: health#nvim_treesitter#check
========================================================================
## Installation
- OK: `tree-sitter` found 0.20.2 (parser generator, only needed for :TSInstallFromGrammar)
- OK: `node` found v17.3.0 (only needed for :TSInstallFromGrammar)
- OK: `git` executable found.
- OK: `gcc-11` executable found. Selected from { "gcc-11", "cc", "gcc", "clang", "cl", "zig" }
Version: gcc-11 (Homebrew GCC 11.2.0_3) 11.2.0
- OK: Neovim was compiled with tree-sitter runtime ABI version 13 (required >=13). Parsers must be compatible with runtime ABI.
...
Still trying to figure it out the issue
Seems like you're on a old SDK version. What is your macOS Version ?
@danymat Catalina, macOS version 10.15.7
Seeing the following compilation error after upgrading to Monterrey:
nvim-treesitter[norg]: Error during compilation
Undefined symbols for architecture x86_64:
"_main", referenced from:
implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [parser.so] Error 1
Same problem
Hello, can you guys check out what's the output of gcc-11 --version
?
It doesn't look like I have gcc-11, but the output of gcc --version
is:
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1 Apple clang version 13.0.0 (clang-1300.0.29.30) Target: x86_64-apple-darwin21.3.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Yeah, you should use gcc version instead. It seems that default gcc version (even if it's gcc-11) fails on Mac
Sorry I'm not sure I follow. So here's what I did.
brew install gcc
/usr/local/bin/gcc-11
to /usr/local/bin/gcc
So that it would be picked up before the system gcc on mac.:TSInstall norg
in a new nvimStill getting the same compilation error. Looking here https://stackoverflow.com/questions/11852568/gcc-4-8-on-mac-os-x-10-8-throws-undefined-symbols-for-architecture-x86-64 it seems like it's not linking the C++ std library by default?
I then symlinked /usr/local/bin/gcc-11 to /usr/local/bin/gcc So that it would be picked up before the system gcc on mac.
Weirdly it seems that the symlink did not work, or your PATH is somehow different. The output of gcc version with brew should yield different verbose text. Try exploring this part and be sure that doing gcc --version shows brew gcc
Sorry, misunderstanding, I did all of those steps after you said to use gcc. Here's the output of gcc --version
after creating the symlink:
❯ gcc --version gcc (Homebrew GCC 11.2.0_3) 11.2.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Can you show me the output of checkhealth nvim_treesitter
So originally it looked like it was still using /usr/bin/gcc. Then after running export CC=/usr/loca/bin/gcc-11
I got this output
nvim_treesitter: health#nvim_treesitter#check
========================================================================
## Installation
- OK: `tree-sitter` found 0.20.4 (parser generator, only needed for :TSInstallFromGrammar)
- OK: `node` found v14.18.3 (only needed for :TSInstallFromGrammar)
- OK: `git` executable found.
- OK: `/usr/local/bin/gcc-11` executable found. Selected from { "/usr/local/bin/gcc-11", "cc", "gcc", "clang", "cl", "zig" }
Version: gcc-11 (Homebrew GCC 11.2.0_3) 11.2.0
- OK: Neovim was compiled with tree-sitter runtime ABI version 13 (required >=13). Parsers must be compatible with runtime ABI.
Unfortunately the error persists :(
A bit more, I tried using g++ as in that SO link and it failed with an error complaining about the -std=c99 flag
in the binding.gyp file
I am also seeing this compilation error on my Mac, though I have not yet upgraded to Monterey (still on Big Sur).
Everything was working fine, did a :PlugUpdate to update all my plugins via vundle, then this showed up when treesitter recompiled all the language parsers.
Interesting, I've also now seen the same error with the latex parser. Maybe an upstream problem with node-gyp?
I can't run gcc-11 as I'm using macOS 13, which throws an ld
error under gcc-11, so I've installed gcc-12
Running CC=gcc-12 nvim
and getting this error:
nvim-treesitter[norg]: Error during compilation
0 0x1031081a0 __assert_rtn + 140
1 0x102f8fa8c mach_o::relocatable::Parser<arm64>::parse(mach_o::relocatable::ParserOptions const&) + 4536
2 0x102f61d38 mach_o::relocatable::Parser<arm64>::parse(unsigned char const*, unsigned long long, char const*, long, ld::File::Ordinal, mach_o::
relocatable::ParserOptions const&) + 148
3 0x102fca4ac ld::tool::InputFiles::makeFile(Options::FileInfo const&, bool) + 1468
4 0x102fcd360 ___ZN2ld4tool10InputFilesC2ER7Options_block_invoke + 56
5 0x1a20b365c _dispatch_client_callout2 + 20
6 0x1a20c70d0 _dispatch_apply_invoke + 224
7 0x1a20b361c _dispatch_client_callout + 20
8 0x1a20c516c _dispatch_root_queue_drain + 684
9 0x1a20c586c _dispatch_worker_thread2 + 164
10 0x1a2236314 _pthread_wqthread + 228
A linker snapshot was created at:
^I/tmp/parser.so-2022-08-18-162321.ld-snapshot
ld: Assertion failed: (_file->_atomsArrayCount == computedAtomCount && "more atoms allocated than expected"), function parse, file macho_relocatab
le_file.cpp, line 2061.
collect2: error: ld returned 1 exit status
Any tips?
@shaunsingh Hm, that's a very weird error. All I can say is that it's unrelated to our TS parser - something at a lower level (presumably on the tree-sitter
side) is messing up, and the assertion error is not giving a lot of detail :/
I know not everyone likes clang, but have you tried it? It could be a gcc oddity for all we know.
@vhyrro Clang fails altogether with the same error as the first comment above (here: https://github.com/nvim-neorg/tree-sitter-norg/issues/7#issue-984407917).
FWIW, I was able to compile the neorg TS parser under MacOS Monterey version 12.5.1 using gcc 12. Attempting to compile with clang, on the other hand, fails with the same errors as in the original message in this thread.
I get same error
4 0x102a9d360 ___ZN2ld4tool10InputFilesC2ER7Options_block_invoke + 56
5 0x1893941f4 _dispatch_client_callout2 + 20
6 0x1893a7954 _dispatch_apply_invoke + 224
7 0x1893941b4 _dispatch_client_callout + 20
8 0x1893a5a04 _dispatch_root_queue_drain + 680
9 0x1893a6104 _dispatch_worker_thread2 + 164
10 0x189554324 _pthread_wqthread + 228
A linker snapshot was created at:
/tmp/smr_dense_SL-2022-09-14-185324.ld-snapshot
ld: Assertion failed: (_file->_atomsArrayCount == computedAtomCount && "more atoms allocated than expected"), function parse, file macho_relocatable_file.cpp, line 2061.
collect2: error: ld returned 1 exit status
make: *** [smr_dense_SL] Error 1
Also getting an error:
Also reported on nvim-neorg/neorg#580
Wrestled with this a lot these past few days. Turns out to be largely bad timing.
I upgraded to the latest XCode 14 and its set of command line tools, but there's something off that breaks other build tools.
The trick was signing up for a free apple dev account, and installing Command Line tools 13.4 from the following:
https://developer.apple.com/download/all/?q=Command%20LIne%20Tools%2013
Then run this command to switch CLI Tools versions
sudo xcode-select --switch /Library/Developer/CommandLineTools
Then run the following to get the version. Make sure it's 13.4.
pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
Additionally if you're using a GUI like Neovide you will likely need to edit your ~/.zprofile file to set the CC env var, as that is still needed to point to gcc-12.
After that :Neorg sync-parsers
worked from a norg file.
Full credit to Magic105 for figuring that out in https://stackoverflow.com/a/73765819/5614103
Thanks for the fix! my macOS gives an error that 13.4 is to old a version for macOS 13, probably explains why macOS 13 gives an error as well but it works on 12
It's unclear to me why the compilation errors can't be fixed when compiling with clang.
If you mean Apple Clang, then you have to explicitly tell it C++ version. It defaults to pre C++11 version.
FYI with clang 15 & 14 installed through MacPorts it went through fine.
MacOS 12.4, had to downgrade to CLT 13.4 and use GCC, so still an issue with Apple-supplied Clang
MacOS 12.6
Installed the Command Line Tools 14.1 RC-2 via https://developer.apple.com/download/all/?q=Command%20LIne%20Tools%2014.1 and GCC via brew install gcc
CC=gcc-12 nvim -c "TSInstall norg"
Update: typo fixed (thanks @kas227) , and see next comment of @kas227 for a possibly needed extra step.
Tyvm @edwardsmit
This almost made it work for me. The only part that was missing after installing the tools was to also switch
, ie.
sudo xcode-select --switch /Library/Developer/CommandLineTools
Also minor typo in you prompt
CC=gcc-12 nvim -c "TSInstall norg"
(nog
instead ofnorg
)
For those on macOS (personally I am on 13.3.1)
The full steps I used to get this working were:
brew install gcc
xcode-select --install
sudo xcode-select --switch /Library/Developer/CommandLineTools
CC=gcc-12 nvim -c "TSInstall norg"
For those on macOS (personally I am on 13.3.1)
The full steps I used to get this working were:
brew install gcc xcode-select --install sudo xcode-select --switch /Library/Developer/CommandLineTools CC=gcc-12 nvim -c "TSInstall norg"
Can someone explain to me why this worked?
Fwiw, the guides here worked for me, but brew install gcc
now installs gcc==13, so I had to change all gcc-12
to gcc-13
.
This worked for me:
brew install gcc@12
CC="/opt/homebrew/bin/gcc-12" nvim
CC="/opt/homebrew/bin/gcc-12" nvim -c "TSInstallSync norg"
I think the issue comes from two places:
1) For some odd reason Apple Clang seems to default to C++98 as its compilation target which breaks the build script for compiling the parser itself.
(scanner.cc compiles with clang++ -std=c++11
on Mac)
2) The C++ in which scanner.cc was written in seems to use a lot of C++11 features compared to other tree sitter parsers which breaks this specific parser where other ones are fine
The closest thing to a fix without needing to install another dependency that I have found is to run:
CC="/usr/bin/clang++ -std=c++17" nvim -c "TSInstallSync norg"
The closest thing to a fix without needing to install another dependency that I have found is to run:
CC="/usr/bin/clang++ -std=c++17" nvim -c "TSInstallSync norg"
This will not work, because when scanning for compilers from CC
env var, treesitter installer will call shell.select_executable(compilers)
which, in its turn, will call vim.fn.executable()
on each entry.
vim.fn.executable('clang++')
= 1vim.fn.executable('clang++ -std=c++17')
= 0This seems to work:
local shell = require 'nvim-treesitter.shell_command_selectors'
local install = require 'nvim-treesitter.install'
-- save the original functions
local select_executable = shell.select_executable
local compilers = install.compilers
-- temporarily patch treesitter install logic
local cc = 'clang++ -std=c++11'
function shell.select_executable(executables)
return vim.tbl_filter(function(c) ---@param c string
return c ~= vim.NIL and (vim.fn.executable(c) == 1 or c == cc)
end, executables)[1]
end
install.compilers = { cc }
-- install norg parsers
install.commands.TSInstallSync['run!'] 'norg' -- or vim.cmd [[ :TSInstallSync! norg ]]
-- restore the defaults back
shell.select_executable = select_executable
install.compilers = compilers
This should be good enough to be placed e.g. in build()
function in lazy.
It's way too complicated though and 100% of macOS users will hit this, so maybe it should be handled by Neorg itself?
@vhyrro What do you think? (it's a bit brittle but still better than the build crashing for everyone on the mac)
All, Installing gcc-12 wasn't enough for me.
This comment worked: https://github.com/nvim-neorg/tree-sitter-norg/issues/7#issuecomment-1251675173
However, there are some gaps in that comment that caused me to stumble into issues. He is my attempt at a guide.
xcode-select --install
brew install gcc@12; which gcc-12
sudo xcode-select --switch /Library/Developer/CommandLineTools
pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
export CC=$(which gcc-12)
nvim tmp.norg
:Neorg sync-parsers
I found that just setting the use_makefile
option to true
under the norg ts parser config did the trick
You should no longer need to install gcc-12
or set CC paths etc: https://github.com/nvim-neorg/neorg/pull/891
I found that just setting the
use_makefile
option totrue
under the norg ts parser config did the trick
hey man, do you mean literally inside the norg parser repo or how did you do this?
I have the following in my vim config
When I run
TSUpdate
, I get this error:From
:checkhealth nvim_treesitter
And
cc --version