tree-sitter-grammars / tree-sitter-markdown

Markdown grammar for tree-sitter
MIT License
411 stars 52 forks source link

bug: Final fence not properly parsed in code blocks #152

Closed pokey closed 3 months ago

pokey commented 3 months ago

Did you check existing issues?

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

No response

Describe the bug

The final fenced delimiter ``` sometimes fails to be recognized as a (fenced_code_block_delimiter) node.

Note that it works correctly if there is a paragraph after the code block. For example, the following:

hello


hello there

parses to:

(document
  (section
    (fenced_code_block
      (fenced_code_block_delimiter)
      (block_continuation)
      (code_fence_content
        (block_continuation)
      )
      (fenced_code_block_delimiter)
    )
    (paragraph
      (inline)
    )
  )
)

as expected

Steps To Reproduce/Bad Parse Tree

Input:

hello

Output:

(document
  (section
    (fenced_code_block
      (fenced_code_block_delimiter)
      (block_continuation)
      (code_fence_content
        (block_continuation)
        "`"
        "`"
        "`"
      )
    )
  )
)

Note the three "`" at the end, which should instead be a (fenced_code_block_delimiter) node.

Expected Behavior/Parse Tree

(document
  (section
    (fenced_code_block
      (fenced_code_block_delimiter)
      (block_continuation)
      (code_fence_content
        (block_continuation)
      )
      (fenced_code_block_delimiter)
    )
  )
)

Repro

No response

pokey commented 3 months ago

ooh actually on closer inspection it looks like you don't even need a trailing paragraph; it suffices to just have a trailing newline on the final fence

pokey commented 3 months ago

...which leads me to realise this is a duplicate of https://github.com/tree-sitter-grammars/tree-sitter-markdown/issues/135