Sending commands from fenced code blocks sends the fence too

[j-w-e]

Running the RDSendLine command within a code block, on Rmd or qmd files, tries to send the entire code block including some of the fence, not just the current line.

For example, in a qmd or rmd file that contains:

1 + 1 # Cursor is on this line

Running \l for example produces Error: object '{r}\n\n1 + 1\n\n' not found.

Tested using this minimal config:

local lazypath = vim.fn.stdpath("data") .. "/lazy/lazy.nvim"
if not (vim.uv or vim.loop).fs_stat(lazypath) then
  vim.fn.system({
    "git",
    "clone",
    "--filter=blob:none",
    "https://github.com/folke/lazy.nvim.git",
    "--branch=stable", -- latest stable release
    lazypath,
  })
end
vim.opt.rtp:prepend(lazypath)

vim.g.mapleader = " "
vim.g.maplocalleader = "\\"

require("lazy").setup({
  { "R-nvim/R.nvim", dependencies = {
    "nvim-treesitter/nvim-treesitter",
  } },
})

[jalvesaq]

I'm noting an inconsistency between what :InspectTree displays and the output of vim.treesitter functions.

With a single line R script (x <- 1), the output of :InspectTree is:

(program ; [0, 0] - [1, 0]
  (left_assignment ; [0, 0] - [0, 6]
    name: (identifier) ; [0, 0] - [0, 1]
    value: (float))) ; [0, 5] - [0, 6]

Then, if the cursor is over the letter x, the output of vim.treesitter functions are:

:lua print(vim.treesitter.get_node({lang = "r"}):type()):

identifier

Correct! x is an identifier.

:lua print(vim.treesitter.get_node({lang = "r"}):parent():type()):

left_assignment

Correct! x is within a left_assignment command.

However, with the following Quarto script:

---
title: "Fenced code bug"
---

```{r}
x <- 1
``` 

The output of :InspectTree is:

(document ; [0, 0] - [7, 0]
  (minus_metadata ; [0, 0] - [3, 0]
    (stream ; [1, 0] - [2, 0]
      (document ; [1, 0] - [2, 0]
        (block_node ; [1, 0] - [2, 0]
          (block_mapping ; [1, 0] - [2, 0]
            (block_mapping_pair ; [1, 0] - [1, 24]
              key: (flow_node ; [1, 0] - [1, 5]
                (plain_scalar ; [1, 0] - [1, 5]
                  (string_scalar))) ; [1, 0] - [1, 5]
              value: (flow_node ; [1, 7] - [1, 24]
                (double_quote_scalar)))))))) ; [1, 7] - [1, 24]
  (section ; [3, 0] - [7, 0]
    (fenced_code_block ; [4, 0] - [7, 0]
      (fenced_code_block_delimiter) ; [4, 0] - [4, 3]
      (info_string ; [4, 3] - [4, 6]
        (language)) ; [4, 4] - [4, 5]
      (block_continuation) ; [5, 0] - [5, 0]
      (code_fence_content ; [5, 0] - [6, 0]
        (program ; [5, 0] - [6, 0]
          (left_assignment ; [5, 0] - [5, 6]
            name: (identifier) ; [5, 0] - [5, 1]
            value: (float))) ; [5, 5] - [5, 6]
        (block_continuation)) ; [6, 0] - [6, 0]
      (fenced_code_block_delimiter)))) ; [6, 0] - [6, 3]

Then, if the cursor is over the letter x, the output of vim.treesitter functions are:

:lua print(vim.treesitter.get_node({lang = "r"}):type()):

identifier

Correct! But maybe the vim.treesitter command should not work because the ignore_injections option is missing.

:lua print(vim.treesitter.get_node({lang = "r"}):parent():type()):

program

Wrong! It should be left_assignment.

:lua print(vim.treesitter.get_node_text(vim.treesitter.get_node({lang = "r"}), 0)):

`{r}
x <- 1
` 

Wrong! This is the correct output of :lua print(vim.treesitter.get_node_text(vim.treesitter.get_node(), 0)), that is, the text of node fenced_code_block.

[jalvesaq]

@PMassicotte, @wurli: It seems to be a bug in nvim-treesitter...

[PMassicotte]

If I am in an Rmd OR a qmd file with cursor on x and type:

:lua print(vim.treesitter.get_node({lang = "r"}):parent():type())

I get program as output.

Here are both trees for Rmd and qmd

---
title: "Fenced code bug"
---

```{r}
x <- 1

### Rmd

(minus_metadata ; [1:1 - 4:0] (document ; [2:1 - 3:0] (block_node ; [2:1 - 3:0] (block_mapping ; [2:1 - 3:0] (block_mapping_pair ; [2:1 - 24] key: (flow_node ; [2:1 - 5] (plain_scalar ; [2:1 - 5] (string_scalar))) ; [2:1 - 5] value: (flow_node ; [2:8 - 24] (single_quote_scalar))))))) ; [2:8 - 24] (section ; [4:1 - 8:0] (fenced_code_block ; [5:1 - 8:0] (fenced_code_block_delimiter) ; [5:1 - 3] (info_string ; [5:4 - 6] (language)) ; [5:5 - 5] (block_continuation) ; [6:1 - 0] (code_fence_content ; [6:1 - 7:0] (left_assignment ; [6:1 - 6] name: (identifier) ; [6:1 - 1] value: (float)) ; [6:6 - 6] (block_continuation)) ; [7:1 - 0] (fenced_code_block_delimiter))) ; [7:1 - 3]

### qmd

(minus_metadata ; [1:1 - 4:0] (document ; [2:1 - 3:0] (block_node ; [2:1 - 3:0] (block_mapping ; [2:1 - 3:0] (block_mapping_pair ; [2:1 - 24] key: (flow_node ; [2:1 - 5] (plain_scalar ; [2:1 - 5] (string_scalar))) ; [2:1 - 5] value: (flow_node ; [2:8 - 24] (double_quote_scalar))))))) ; [2:8 - 24] (section ; [4:1 - 8:0] (fenced_code_block ; [5:1 - 8:0] (fenced_code_block_delimiter) ; [5:1 - 3] (info_string ; [5:4 - 6] (language)) ; [5:5 - 5] (block_continuation) ; [6:1 - 0] (code_fence_content ; [6:1 - 8:0] (left_assignment ; [6:1 - 6] name: (identifier) ; [6:1 - 1] value: (float)) ; [6:6 - 6] (identifier) ; [7:1 - 2] (identifier) ; [7:3 - 4] (block_continuation)))) ; [7:1 - 0]

[PMassicotte]

I do not understand why don't we have the same output…

[PMassicotte]

Sending the current line is not working on my side either. I am getting:

[ins] r$> `{r}

          x <- 1

          `
Error: object '{r}\n\nx <- 1\n\n\n' not found

However, sending code chunk <localleader>cc works fine.

[wurli]

I’ll take a look at this tonight… Initial thoughts:

• I wonder if <localleader>cc will behave differently before/after require("r.send”).send(), since send() injects the R parser for the current code chunk.

• Could any differences in behaviour between your setups be due to differing versions of nvim-treesitter?

[jalvesaq]

• I wonder if <localleader>cc will behave differently before/after require("r.send”).send(), since send() injects the R parser for the current code chunk.

<LocalLeader>cc runs a code directly converted from Nvim-R, with no use of treesitter.

• Could any differences in behaviour between your setups be due to differing versions of nvim-treesitter?

I'm running the current development version of Neovim from Git Hub. My nvim-treesitter configuration is:

    {
        "nvim-treesitter/nvim-treesitter",
        tag = nil,
        branch = "master",
        run = ":TSUpdate",
        config = function ()
            require("nvim-treesitter.configs").setup({
                sync_install = true,
                ensure_installed = {
                    "bash",
                    "c",
                    "css",
                    "html",
                    "json",
                    "latex",
                    "lua",
                    "markdown",
                    "markdown_inline",
                    "python",
                    "query",
                    "r",
                    "rnoweb",
                    "vim",
                    "vimdoc",
                    "yaml",
                },
                highlight = {
                    enable = true,
                },
            })
            vim.o.foldmethod = "expr"
            vim.o.foldexpr = "nvim_treesitter#foldexpr()"
            vim.o.foldenable = false
        end
    },

[jalvesaq]

I get program as output.

Note that there is no node of type "program" in your :InspectTree outputs...

[jalvesaq]

The bug is fixed for me on the branch send_chunk_line. Tested with:

Rnoweb:

\documentclass{article}

\begin{document}

<<>>=
#| label: foo
#| echo: true
#| results: hide

library(dplyr)
iris |>
  subset(Species == "setosa") |>
  filter(Sepal.Length > 5) |>
  transform(Sepal.Area = Sepal.Length * Sepal.Width)

summary(iris,
        digits = 4)
@

<<>>=
#| label: bar
#| echo: false

x <- rnorm(10)
print(x)
@

\end{document}

Quarto/Rmd:

```{r}
#| label: foo
#| echo: true
#| results: hide

library(dplyr)
iris |>
  subset(Species == "setosa") |>
  filter(Sepal.Length > 5) |>
  transform(Sepal.Area = Sepal.Length * Sepal.Width)

summary(iris,
        digits = 4)
```

```{r}
#| label: bar
#| echo: false

x <- rnorm(10)
print(x)
```

[jalvesaq]

Notes:

• The function ensure_ts_parser_exists() was unnecessary in the scripts above (Rnoweb and Quarto).
• We need the option lang = "r" for Rnoweb, but not for Quarto/Rmd. Indeed, the output of :InspectTree shows the typical R nodes in Rmd/Quarto, but not in Rnoweb.
• I simplified the code to find the root node, but there was no real change there.
• Unrelated to this bug fix, I deleted a code that was used in Nvim-R but not in R.nvim (find("^The topic")).

[PMassicotte]

Good job!

[wurli]

Many thanks! One question from me if you don't mind. It looks like the language for the treesitter parser is now being manually set to R only for Rnoweb buffers. How does treesitter know to parse Quarto/Rmd chunks as R? And how does it know that text outside of the chunk is not R code? I was previously under the impression that it wasn't smart enough to figure this out without some help. Is this configured elsewhere? Of course it's highly possible my previous understanding was simply incorrect 😁

[jalvesaq]

What I did was more the result of trial and error than understanding, but, if we do :InspectTree in Rmd/Quarto and in Rnoweb files, we see a clear difference: the R node names are present in the Rmd/Quarto tree, but not in the Rnoweb tree.

Initially, I was trying an old Rnoweb file with the knitr options in the chunk header, and there was no syntax highlight for the R code. But sending code R worked! Accidentally (when typing a pipe operator), I created a chunk without knitr options and discovered that rnoweb indeed can highlight R code. Then, I moved the kntir options to the lines below the chunk header, as is usual in Quarto documents, and the sending code stopped working. That is, I replicated in Rnoweb the bug that I was seeing only in Rmd/Quarto. Conclusion: manually setting the R parser was required only if the default document parser didn't recognize R code. At this point, I started experimenting with enabling and disabling some vim.treesitter.get_node() options until it worked in all three cases: R, Rmd/Quarto, and Rnoweb.

[jalvesaq]

I turned nvim-treesitter a mandatory dependency in the send_chunk_line branch.

@j-w-e could you try the send_chunk_line branch, please?

[j-w-e]

Yes, this seems to work on initial testing. Thank you, for this and all the work you've put into the plugin in general.

[jalvesaq]

Thank you for the feedback!

[wklimowicz]

I just spotted the same issue for paragraph send which is my daily driver (:lua require('r.send').paragraph(true) on the chunk below sends the backticks to the console)

```r
1 + 1   # Cursor is on this line

Interestingly this is new with the lua refactor, it works fine in Nvim-R. Let me know if I should open a new issue, but I wouldn't know where to start fixing it.

[jalvesaq]

It's a different bug, but it's fixed now.