Treesitter R

[inventionate]

Not really a problem, but more of a hint. An R parser for Neovim Treesitter will be available soon. I have already tried it and it seems to be very good. Maybe someone is interested in it.

https://github.com/nvim-treesitter/nvim-treesitter/pull/1012

And perhaps in the future, when Treesitter is more stable and Neovim 0.5 has been released, one can think about using this system for syntax handling.

Once again thank you very much for your great plugin!

[jalvesaq]

Thanks for the hint. The current syntax scripts are hard to write and maintain, and occasionally the correct highlighting is lost. It would be great if someone could replace them with something better and easier to maintain. Are there other benefits of a Treesitter parser in addition to better and faster syntax highlighting?

[inventionate]

Thank you for your answer and interest. I've been using Treesitter for a few weeks so far and, as I said, I'm very fond of it. In fact, there are some other advantages. For example, there is the possibility to implement so-called modules (https://github.com/nvim-treesitter/nvim-treesitter#available-modules). Currently Treesitter R supports mainly syntax highlighting and incremental selection. However, there is also the possibility to create modules for e.g. indention and folding. I think these would be very big advantages and simplifications, because the current code is—as you pointed out—probably quite complex. There is also the possibility to use Treesitter for code completion (https://github.com/nvim-treesitter/completion-treesitter), but for this I think the R Languageserver (https://github.com/REditorSupport/languageserver) in combination with the native LSP client of Neovim 0.5 (https://neovim.io/doc/user/lsp.html) and good old classic omnicompletion (which can be integrated in the LSP client) is more advantageous. I hope this shows some possibilities and allows for future support of Treesitter someday.

[jalvesaq]

The treesitter system of parsing the code is very promising. Being aware of the structure of the code is essential for proper highlighting in some special cases, and I hope it will fix the indentation bugs too. Below is a comparison of the same code highlighted in three different ways: (1) Treesitter disabled, but with Neovim + Nvim-R traditional highlighting. (2) Treesitter highlighting over traditional highlighting. (3) Only the Treesitter highlighting (I removed Nvim-R and V-R-runtime, and deleted syntax/r.vim from Neovim runtime directory).

Things that I see that could be improved are:

• The highlight of the NOTE was messed up in (2). The treesitter replaced the foreground color, but not the background one.
• The arguments first and second are highlighted only in the function header, not in its body.
• Special characters within strings are not recognized.
• TRUE and FALSE should be highlighted as Boolean and not as Keyword.
• The string "abc" is only underlined. The color indicates the type of spelling mistake, but it was lost.
• In (3), words not in comments or strings are marked as misspelled words.
• The Treesitter is ignoring the option r_syntax_fun_pattern whose default value is 0 if Nvim-R is installed. That is, keywords followed by an opening parenthesis should not be highlighted as functions, and only functions of loaded packages should be highlighted as keywords.
• Because Treesitter knows the structure of the code, I think it should highlight as error the nothing() above the function definition because R would not found it if the code was run.
• Similarly, the keyword fread should be highlighted as a function only after library(data.table) because only then it would be available.

[inventionate]

I've linked your suggestions to the corresponding Treesitter R package. Maybe there is a possibility to improve the grammar. In any case, thank you very much for the detailed suggestions!

I have also discovered another interesting possibility that Treesitter opens up:

https://github.com/weilbith/nvim-treesitter-context

[lf-araujo]

@inventionate thanks to pointing to these sources, treesitter seems to be working ok in R. However, I could never find a sensible neovim's standard lsp configuration for R. Have you found any?

[inventionate]

@lf-araujo you're welcome. Yes, I found a good solution. You can check out my neovim configuration here: https://gitlab.com/Inventionate/dotfiles/-/blob/main/init.vim the relevant part is located in a separate lua file here: https://gitlab.com/Inventionate/dotfiles/-/blob/main/nvim/lua/config/lsp.lua

Basically, you just have to install the languageserver package from CRAN and use the https://github.com/neovim/nvim-lspconfig plugin (https://github.com/neovim/nvim-lspconfig/blob/master/CONFIG.md#r_language_server) with the recent neovim 0.5 and everything should be fine. Since the integrated LSP client is very low-threshold, I recommend using a few more tweaks and plug-ins. You can find my preferences in my dotfiles. If something is unclear or causes problems, feel free to contact me. I hope this helps you.

[lf-araujo]

Thank you so much! It indeed works as intended for R files, but how to activate it in rmd files? Did you find a solution for this @inventionate ?

[inventionate]

Yes, I use MUcomplete and configure my own completion chain. Basically I use both, the Nvim-R (for cite completion) and languageserver omnicompletion. You can find it in my init.vim The relevant code is

let g:mucomplete#always_use_completeopt = 1
let g:mucomplete#chains = {}
let g:mucomplete#chains = {
            \ 'default' : ['omni', 'path', 'c-n'],
            \ }
let g:mucomplete#chains['rmd'] = {
            \ 'default' : ['user', 'path'],
            \ 'rmdrChunk' : ['omni', 'path'],
            \ }

autocmd FileType rmd set completefunc=pandoc#completion#Complete

If you won’t use MUcomplete it’s also possible. You just have to overwrite Nvim-R omnicompletion and use lsp instead. You can find the code for that in the lsp.lua file. Because I prefer completion on tab I use a sync lsp omnicompletion function. If you want autocompletion while you type you can use compe and the built in async lsp function. Or you just use the built in vim omnicompletion ctrl_x ctrl_o without any plugin. Hope this helps.

[lf-araujo]

Sorry if this is hijacking the Nvim-R support, but I think this might be useful to other people that are interested in migrating to treesitter and lsp. This complements Nvim-R very well.

One final q @inventionate I noticed that treesitter is not starting for rmd files, you can check this with:

:lua print(vim.inspect(vim.treesitter.get_parser(0)))

• Did you notice this too?

[inventionate]

Yes, at the moment there is no stable treesitter markdown parser and so treesitter for r markdown is (at the moment) not available. If you take a look at the treesitter repo you'll see there is an ongoing discussion and work around that (especially in the case of pandoc markdown). Anyway, I hope and think there will be a treesitter markdown parser in the future.

[BlackEdder]

Yes, at the moment there is no stable treesitter markdown parser and so treesitter for r markdown is (at the moment) not available.

It looks like there might finally be a stable treesitter for markdown: https://github.com/nvim-treesitter/nvim-treesitter/issues/872#issuecomment-992857459 .

[sebastianbock17]

Is it thus now possible to use treesitter with rmarkdown, and how? Sorry for this stupid question.. I also like nvim-r and am trying to get syntax highlighting to work with it, but do not yet understand how..

Kind regards :)

[jalvesaq]

I'm not following the development of treesitter in Neovim, but I can merge pull requests into R-Vim-runtime if someone writes them.

[kofm]

I think it is unfortunately not possible at the moment because there's no available RMarkdown tree sitter parser. Even though a markdown parser is now available and can be forked to implement this, there's still no one available as far as I know. I tried to fork the repo myself to give it a try but is way beyond my actual programming skills :( R scripts work fine though..

[lf-araujo]

Hi all, it does work!

Run :TSInstall markdown and it gets picked up normally both in .R and .Rmd files. Works pretty well with Nvim-R. I am on mobile, so here is a link to my config instead of the pasted code: https://gist.github.com/lf-araujo/9b3e239757d6f48a564d75818b46f17c

Good luck.

[kofm]

I don't understand. What is working? With nvim-treesitter and the markdown parser installed and updated to newest version only markdown files get highlighted, as expected. If I force markdown filetype on a rmd file using :set filetype=markdown, i get a bunch of errors, then some of the markdown bits are highlighted but the code within the chunks is not. Maybe the highlight you're seeing is coming from Nvim-R itself?

R script files are correctly highlighted thanks to tree-sitter-r parser

[inventionate]

As far as I know, it is currently still not possible to use a Treesitter Markdown parser for R Markdown. The current Markdown parser is in an early stage of development and also targets CommonMark, not Pandoc Markdown. Also, the R parser would have to be implemented for code chunks, which is also still a long way off (and as far as I can see not currently intended from anybody). I think at the moment Treesitter works very well for R scripts, especially since the recent updates. For R Markdown files, on the other hand, the syntax implementation in Nvim-R is much better (and I share the suspicion that it is what works).

[sebastianbock17]

Thank you @lf-araujo, I will definately check it out!! (without the windows mappings :) ).

Also, the R parser would have to be implemented for code chunks, which is also still a long way off (and as far as I can see not currently intended from anybody)

My question regards exactly the code chunks, as a different style of code fenced blocks.

Because I use other markdown extensions, such as vimwiki side by side in rmarkdown files, it is totally understandable that this is Not what is intended.

So a simple solution would just be a quick :set filetype r to work with the R code, as I look upon the code chunks while writing, disregarding the code chunks, ({r}) or would this not be possible and still only for r?

Just the simple {r} instead of r workaround would be fine for me, while leaving other options for code chunks completely out (including refs).

For R Markdown files, on the other hand, the syntax implementation in Nvim-R is much better (and I share the suspicion that it is what works).

Thanks @inventionate for your insight. Do you have an idea how I might could be using nvim-r's syntax highlighting for rmarkdown, and at times switching to a more custom markdown syntax highlighting instead, like vimwiki? This should be doable with ease, or am I wrong?

So I will start from these current best practices, trying out and customising @lf-araujo's vimrc and if it should really not work, using the standard nvim-r syntax highlighting again for .rmd - which I also have as very good in my memory and quite important when e.g. referencing tables in the paper :)

[lf-araujo]

Hey all,

I am not sure why you are not getting treesitter started on rmd files. Just tested in a separate nvim distribution (LunarVim) and the chunks are highlighted after TS is activated, provided TSInstall markdown is installed. Also my init.vim stopped working for whatever change on upstream lsp changes, so my current recommendation is to math/stats/sciency peoples install LunarVim, follow instructions here, and add the following to the configuration (~/.config/lvim/config.lua):

-- Additional Plugins
lvim.plugins = {
    {"jalvesaq/Nvim-R"}
}

And you will have the same results as my own old config.

Edit: caused a lot of confusion, sorry: :TSInstall markdown

[kofm]

I'm sorry but I do not understand.

If i try to run :TSInstall rmarkdown i get an error because there's no available rmarkdown parser. The instructions you linked are related to installing Lsp for R which is working nicely out of the box for both R and Rmarkdown files but does not provide any highlight. To verify that Treesitter is actually highlighting your rmarkdown files can you please install tree sitter playground and run :TSPlaygroundToggle with a RMD file open?

It should show a tree like this one if it's working:

[sebastianbock17]

I also followed the instruction for using lvim, when running :TSInstall rmarkdown, I get:

E5108: Error executing lua ...er/start/nvim-treesitter/lua/nvim-treesitter/install.lua:373: Parser not available for language rmarkdown stack traceback: [C]: in function 'get_parser_install_info' ...er/start/nvim-treesitter/lua/nvim-treesitter/install.lua:373: in function 'install_lang' ...er/start/nvim-treesitter/lua/nvim-treesitter/install.lua:422: in function 'run' [string ":lua"]:1: in main chunk

…

But all the code chunks are recognized .rmd, so I do get highlighting, the snippets from nvim-r working, I am not sure how playground works yet though, as the playground buffer is just empty on my side

[lf-araujo]

Sorry, my bad. My above comment should read :TSInstall markdown (not rmarkdown)

[sebastianbock17]

Okay thanks a lot for the suggestion!

The highlighting seems good, however not as good as in R files, when I open some big .rmd files the colors switch from blue to orange. User defined R objects not in the code chunks are not recognized. But markdown elements, yaml, also citations, are recognized.

Maybe this is not perfect, but it's already more than I asked for. So many thanks @lf-araujo, greetings from Germany !! :)

[inventionate]

Thanks for the notes and I'm glad it's sufficient for some scenarios. As I wrote before, the current treesitter markdown parser will never support R Markdown 100 %. It would have to support Pandoc for that, which is not going to happen. However, perhaps a good variant will emerge on this basis in the future, but it would have to be developed. The parser developer also points this out: https://github.com/MDeiml/tree-sitter-markdown/issues/22

[kofm]

But all the code chunks are recognized .rmd, so I do get highlighting, the snippets from nvim-r working, I am not sure how playground works yet though, as the playground buffer is just empty on my side

Playground is empty because the highlight you're seeing are provided by NvimR and not treesitter. Nothing wrong with that, the highlights provided are perfect. The advantage with treesitter would be the use of textobjects like, e.g. in plain R script with tree-sitter-r parser I can incrementally select (and then execute) only parts of a pipe (which is very very useful for devugging), or swap the arguments of a function, or get smart pairing of parentheses, ecc.

[aspeddro]

https://github.com/MDeiml/tree-sitter-markdown/issues/33 added support for code chunks.

-- extend markdown parser to rmd files
local ft_to_parser = require('nvim-treesitter.parsers').filetype_to_parsername
ft_to_parser.rmd = 'markdown'

[dmnkf]

-- extend markdown parser to rmd files
local ft_to_parser = require('nvim-treesitter.parsers').filetype_to_parsername
ft_to_parser.rmd = 'markdown'

I tried this @aspeddro, but it still doesn't work. The syntax itself is not even starting.

enabling it just leads back to the default md highlighting:

It appears as if the treesitter is not even running (check with :lua print(vim.inspect(vim.treesitter.get_parser(0))) ):

E5108: Error executing lua ...r/share/nvim/runtime/lua/vim/treesitter/language.lua:25: no
 parser for 'rmd' language, see :help treesitter-parsers
stack traceback:
        [C]: in function 'error'
        ...r/share/nvim/runtime/lua/vim/treesitter/language.lua:25: in function 'require_
language'
        /usr/share/nvim/runtime/lua/vim/treesitter.lua:39: in function '_create_parser'
        /usr/share/nvim/runtime/lua/vim/treesitter.lua:94: in function 'get_parser'
        [string ":lua"]:1: in main chunk

due to not finding a parser for 'rmd'. The markdown parser is installed tho. Same with R.

Could you perhaps provide the whole init.file for the code you sent to see where it belongs, maybe I did something wrong with my setup.

Here is mine:

local status_ok, nvim_treesitter = pcall(require, 'nvim-treesitter.configs')
if not status_ok then
  return
end
-- extend markdown parser to rmd files
local ft_to_parser = require('nvim-treesitter.parsers').filetype_to_parsername
ft_to_parser.rmd = 'markdown'

-- See: https://github.com/nvim-treesitter/nvim-treesitter#quickstart
nvim_treesitter.setup {
  -- A list of parser names, or "all"
  ensure_installed = {
    'bash', 'markdown', 'c', 'cpp', 'css', 'html', 'javascript', 'json', 'lua', 'python',
    'r', 'rust', 'typescript', 'vim', 'yaml',
  },
  -- Install parsers synchronously (only applied to `ensure_installed`)
  sync_install = false,
  highlight = {
    -- `false` will disable the whole extension
    enable = true,
  },
}

I also tried putting your code after the config setting, because maybe there is some initialization logic I don't understand, but the result stays the same.

EDIT:

Well, it appears as if the syntax trigger is quite case-sensitive. Changing the capital {R} to an {r} fixed the issue. Why there was a capital R to begin with? I do not have the answer to that question.

[kofm]

I don't think {R} is the valid syntax. See https://rmarkdown.rstudio.com/lesson-3.html Also, do not forget to :TSInstall markdown_inline.

[tinyheero]

For posterity purposes,

-- extend markdown parser to rmd files
local ft_to_parser = require('nvim-treesitter.parsers').filetype_to_parsername
ft_to_parser.rmd = 'markdown'

filetype_to_parsername is not deprecated. The new way is to use:

vim.treesitter.language.register("markdown", "rmd")

[jalvesaq]

I'm trying syntax highlighting and indenting from nvim-treesitter. Now, I understand that R-Vim-runtime is for the legacy syntax and indent scripts while Neovim is replacing them with tree-sitter.

So far, I couldn't find any issues with syntax highlighting. The problems that I mentioned in my first comment either are already fixed or should be fixed by a language server, and not by a tree-sitter. I had only to add highlighting for code block headers in Quarto (I copied it from quarto-nvim-kickstarter):

~/.config/nvim/after/queries/r/highlights.scm:

; extends

; From quarto-nvim-kickstarter, YAML header for code blocks,
; but `environment` instead of `literal`
(
(comment) @comment
(#match? @comment "^\\#\\|")
) @text.environment

; Cell delimiter for Jupyter, although I don't use it
(
(comment) @content
(#match? @content "^\\# ?\\%\\%")
) @tag.delimiter

I opted for completely concealing the code block delimiter in Quarto and using headlines.nvim to highlight the code block background. I also added highlighting for div delimiters and for the include command:

~/.config/nvim/after/queries/markdown/highlights.scm:

; extends

;; Conceal curly braces in code block delimiters
(fenced_code_block 
  (fenced_code_block_delimiter)
  (info_string) @conceal
  (#set! conceal ""))

;; Highlight div delimiters
(
(paragraph) @paragraph
(#match? @paragraph "^:::")
) @tag.delimiter

;; Highlight include command
(
(paragraph) @paragraph
(#match? @paragraph "^\\{\\{\\< ")
) @include

The indentation, however, seems to be incomplete, but with the following configuration, it's better in my opinion:

~/.config/nvim/after/queries/r/indents.scm:

;; extends

(left_assignment) @indent.begin

(if (call)) @indent.begin

(if (left_assignment)) @indent.begin

(if (identifier)) @indent.begin

(if (for)) @indent.begin

(for (if)) @indent.begin

(for (call)) @indent.begin

(for (for)) @indent.begin

(for (left_assignment)) @indent.begin

[jalvesaq]

Considering that new features are no longer being developed for Nvim-R, I'm closing this "issue".

New features using tree-sitter should be requested at R-nvim/R.nvim, preferably as a pull request.