Astisme commented 1 year ago

Prework

I followed the steps in #5 to make the apex, soql and sosl highlighting available. Here's my file tree (on Windows 11):

$CONFIG/
|  nvim/
|  |  init.lua (sources setup.vim)
|  |  setup.vim (I've added the augroup FileTypeGroup here)
|  |  lua/astisme/
|  |  |  init.lua (requires packer.lua and nvim-treesitter.lua in this order)
|  |  |  packer.lua (imports nvim-treesitter)
|  |  |  nvim-treesitter.lua (shown below)
|  nvim-data/
|  |  site/pack/packer/start/nvim-treesitter/queries/
|  |  |  apex/
|  |  |  |  *.scm (from this repo /apex/queries/*)
|  |  |  sosl/
|  |  |  |  *.scm (from this repo /sosl/queries/*)
|  |  |  soql/
|  |  |  |  *.scm (from this repo /soql/queries/*)
|  |  tree-sitter-sfapex/ (this repo)
|  |  [other tree-sitter-x folders]

This is my nvim-treesitter.lua

require 'nvim-treesitter.install'.prefer_git = false
require 'nvim-treesitter.install'.compilers = { "clang" }
-- these settings above are required for nvim-treesitter to work on windows

local parser_config = require "nvim-treesitter.parsers".get_parser_configs()
local installation_path = "$CONFIG\\nvim-data\\tree-sitter-sfapex"

parser_config.apex = {
  install_info = {
    url = installation_path .. "\\apex",
    files = {"src/parser.c"},
    branch = "main", -- default branch in case of git repo if different from master
    generate_requires_npm = false, -- if stand-alone parser without npm dependencies
    requires_generate_from_grammar = true, -- if folder contains pre-generated src/parser.c
  }
}

parser_config.soql = {
  install_info = {
    url = installation_path .. "\\soql",
    files = {"src/parser.c"},
    branch = "main", -- default branch in case of git repo if different from master
    generate_requires_npm = false, -- if stand-alone parser without npm dependencies
    requires_generate_from_grammar = true, -- if folder contains pre-generated src/parser.c
  }
}

parser_config.sosl = {
  install_info = {
    url = installation_path .. "\\sosl",
    files = {"src/parser.c"},
    branch = "main", -- default branch in case of git repo if different from master
    generate_requires_npm = false, -- if stand-alone parser without npm dependencies
    requires_generate_from_grammar = true, -- if folder contains pre-generated src/parser.c
  }
}

require 'nvim-treesitter.configs'.setup {
        ensure_installed = {"typescript","html","css","java","bash","javascript","json","python","tsx","lua","vim","apex","sosl","soql"},
                -- one of "all", "maintained" (parsers with maintainers), or a list of languages
        sync_install = false;
                -- Install parsers on demand
        highlight = {
                enable = true,
                        -- false will disable the whole extension
                additional_vim_regex_highlighting = false,
                        -- extra syntax highlighting
        },
}

Research

Checked health on nvim-treesitter

nvim-treesitter: require("nvim-treesitter.health").check()
========================================================================
## Installation
  - OK: `tree-sitter` found 0.20.7 (b268e412ad4848380166af153300464e5a1cf83f) (parser generator, only needed for :TSInstallFromGrammar)
  - OK: `node` found v19.3.0 (only needed for :TSInstallFromGrammar)
  - OK: `git` executable found.
  - OK: `clang` executable found. Selected from { "clang" }
    Version: /usr/bin/bash: line 1: clang --version: command not found
  - OK: Neovim was compiled with tree-sitter runtime ABI version 14 (required >=13). Parsers must be compatible with runtime ABI.

## OS Info:
{
  machine = "x86_64",
  release = "10.0.22621",
  sysname = "Windows_NT",
  version = "Windows 10 Pro" // I know this is weird. Trust me, I have Windows 11.
}

## Parser/Features H L F I J
  - c              ✓ ✓ ✓ ✓ ✓
  - bash           ✓ ✓ ✓ . ✓
  - java           ✓ ✓ . ✓ ✓
  - javascript     ✓ ✓ ✓ ✓ ✓
  - typescript     ✓ ✓ ✓ ✓ ✓
  - tsx            ✓ ✓ ✓ ✓ ✓
  - json           ✓ ✓ ✓ ✓ .
  - python         ✓ ✓ ✓ ✓ ✓
  - lua            ✓ ✓ ✓ ✓ ✓
  - vim            ✓ ✓ ✓ . ✓
  - html           ✓ ✓ ✓ ✓ ✓
  - css            ✓ . ✓ ✓ ✓

Software version

nvim 0.8.1
tree-sitter 0.20.7
npm 9.2.0
clang 15.0.5
Problem

With this setup, I've run :TSInstall apex (and the others) and the output is only
```
[nvim-treesitter] [1/6] Generating source files from grammar.js
```
and it stays stuck at this spot forever.

Tldr

On Windows 11, I cannot make this repo work even though I followed the steps described in #5. nvim-treesitter gets stuck at "generating source files from grammar.js"

xixiaofinland commented 1 year ago

not sure if you still have the problem, I get it working in nvim https://github.com/xixiaofinland/dotfiles/blob/main/.config/nvim/lua/plug-settings.lua#L42

aheber commented 1 year ago

@Astisme, @xixiaofinland has been doing a lot of work on this and was able to get Apex, SOQL, and SOSL registered correctly into the regular nvim-treesitter repo. You should be able to engage with Apex like you would any other properly supported language now.

Please take a look at see if you're able to get that working now.

xixiaofinland commented 1 year ago

@aheber I can create a md to record how to configure in nvim to get it up and running. Shall I create a new md file in this repo and you link it from README, part of README, or in a different repo?

aheber commented 1 year ago

@xixiaofinland please submit as a separate MD file. We can add a link from the README to the other doc so people are aware of it. If there are existing nvim-treesitter docs please link out to them as much as is reasonable. This will help to make sure our docs don't fall out of date.

Thanks!

Astisme commented 1 year ago

Well it's now working. I'd like to report what is the solution as of right now.

Clone the repo

First you should clone the repo in $CONFIG/nvim-data/tree-sitter-sfapex ($CONFIG is ~/AppData/Local)

Update lua script

This is the new nvim-treesitter.lua I use atm.

require 'nvim-treesitter.install'.compilers = { "clang" }

local parser_config = require "nvim-treesitter.parsers".get_parser_configs()
local installation_path = "$CONFIG\\nvim-data\\tree-sitter-sfapex" --$CONFIG from C:\\

parser_config.apex = {
  install_info = {
    url = installation_path .. "\\apex", -- the next comment will explain why not to use link here
    files = {"src/parser.c"},
    branch = "main", -- default branch in case of git repo if different from master
    generate_requires_npm = false, -- if stand-alone parser without npm dependencies
    requires_generate_from_grammar = false, -- if folder contains pre-generated src/parser.c
  }
}
parser_config.soql= {
  install_info = {
    url = installation_path .. "\\soql",
    files = {"src/parser.c"},
    branch = "main", -- default branch in case of git repo if different from master
    generate_requires_npm = false, -- if stand-alone parser without npm dependencies
    requires_generate_from_grammar = false, -- if folder contains pre-generated src/parser.c
  }
}
parser_config.sosl= {
  install_info = {
    url = installation_path .. "\\sosl",
    files = {"src/parser.c"},
    branch = "main", -- default branch in case of git repo if different from master
    generate_requires_npm = false, -- if stand-alone parser without npm dependencies
    requires_generate_from_grammar = false, -- if folder contains pre-generated src/parser.c
  }
}

require 'nvim-treesitter.configs'.setup {
    ensure_installed = {"apex", "soql","sosl"},
    sync_install = false;
    highlight = {
        enable = true,
        additional_vim_regex_highlighting = false,
    },
}

Then restart nvim to install or use :TSInstall apex, :TSInstall soql and :TSInstall sosl. This works without any issues and successfully highlights the apex code (but not the SOQL or SOSL queries) in .cls and .apex files.

Astisme commented 1 year ago

Why cloning is better

Here's a screenshot of the files downloaded. The first is the folder created by tree-sitter; the last is from a git clone. As you can see from this picture, as of right now, if you give tree-sitter the link to the repo, the files downloaded are less then the ones actually available.

I believe it should only be a release problem which could be fixed later, right?

xixiaofinland commented 1 year ago

No, you don't need to do anything other than telling nvim-treesitter to get apex parser installed, like here in my dotfile.

Also, the built-in version has query issue you mentioned resolved for nvim.

Speaking of the file discrepancy, nvim-treesitter technically needs only parser.c, as long as it works, it doesn't matter.

I have an instruction readme PR to be merged: https://github.com/aheber/tree-sitter-sfapex/pull/17/files

@Astisme

xixiaofinland commented 1 year ago

@xixiaofinland please submit as a separate MD file. We can add a link from the README to the other doc so people are aware of it. If there are existing nvim-treesitter docs please link out to them as much as is reasonable. This will help to make sure our docs don't fall out of date.

Thanks!

I made a PR, get the chance to review?

aheber commented 1 year ago

I didn't realize I had left it open. I just merged it.

Astisme commented 1 year ago

@xixiaofinland ok, it is working actually without all the parser_config rows I wrote before. Case closed!

aheber / tree-sitter-sfapex

TreeSitter stuck generating files from grammar #10

Prework

Research

Checked health on nvim-treesitter

Software version

Problem

Tldr

Clone the repo

Update lua script

Why cloning is better