stsewd / tree-sitter-rst

reStructuredText grammar for tree-sitter
https://stsewd.dev/tree-sitter-rst/
MIT License
50 stars 7 forks source link

Non-local trailing whitespace sensitivity. #21

Closed Carreau closed 2 years ago

Carreau commented 2 years ago

Take the following that have no trailing whitespace.

::

    [[1., 0., 0.],
     [0., 1., 2.]]

::

    >>> np.set_printoptions(threshold=sys.maxsize)  # sys module should be imported

You could also implement the reduce as follows::

    >>> def ufunc_reduce(ufct, *vectors):

.. seealso::
   :doc:`numpy-for-matlab-users`

It parses to

document [0, 0] - [15, 0]
  literal_block [2, 4] - [3, 18]
  literal_block [7, 4] - [7, 83]
  paragraph [9, 0] - [11, 41]
    literal_block [11, 4] - [11, 41]
  directive [13, 0] - [13, 12]
    name: type [13, 3] - [13, 10]
  paragraph [14, 3] - [14, 32].  # this is wrong
    interpreted_text [14, 3] - [14, 32]
      role [14, 3] - [14, 8]

at 1 trailing whitespace after the first literal block:

::

    [[1., 0., 0.],
     [0., 1., 2.]]
 <- there is a whitespace here now
::

    >>> np.set_printoptions(threshold=sys.maxsize)  # sys module should be imported

You could also implement the reduce as follows::

    >>> def ufunc_reduce(ufct, *vectors):

.. seealso::
   :doc:`numpy-for-matlab-users`

And it parses properly.

document [0, 0] - [15, 0]
  literal_block [2, 4] - [3, 18]
  literal_block [7, 4] - [7, 83]
  paragraph [9, 0] - [11, 41]
    literal_block [11, 4] - [11, 41]
  directive [13, 0] - [14, 32]
    name: type [13, 3] - [13, 10]
    body: body [13, 12] - [14, 32]
      content [14, 3] - [14, 32]

Note: taken from there https://numpy.org/doc/stable/user/quickstart.html

https://user-images.githubusercontent.com/335567/144559626-0b1aeb7f-968d-4be3-99f3-9859f6e35abd.mov

Carreau commented 2 years ago

note that

 .. seealso::
-   :doc:`numpy-for-matlab-users`
+    :doc:`numpy-for-matlab-users`

Fixes it as well, and so does adding trailing space on other blanks lines like the line just above see-also

I'm not sure this is intended, and a "it should be consistently indented 3/4 spaces" is a valid answer,

stsewd commented 2 years ago

Thanks for reporting! I think this is a bug with the stack that keeps track of indentation levels

https://github.com/stsewd/tree-sitter-rst/blob/a5514617ae3644effa80d4696be428e4a371c01a/src/tree_sitter_rst/parser.c#L903-L907

probably it shouldn't be +1, but the actual indentation level of the block, or maybe there is a dedent missing.

I'm on vacations now, so I'll put some time to look at the bugs in the next days :D

Carreau commented 2 years ago

No worries, at some point I should also delve into tree sitter and understand how it works, so that I can submit patches here directly when I encounter issues.

On Mon, Dec 20, 2021, 16:47 Santos Gallegos @.***> wrote:

Thanks for reporting! I think this is a bug with the stack that keeps track of indentation levels

https://github.com/stsewd/tree-sitter-rst/blob/a5514617ae3644effa80d4696be428e4a371c01a/src/tree_sitter_rst/parser.c#L903-L907

probably it shouldn't be +1, but the actual indentation level of the block, or maybe there is a dedent missing.

I'm on vacations now, so I'll put some time to look at the bugs in the next days :D

— Reply to this email directly, view it on GitHub https://github.com/stsewd/tree-sitter-rst/issues/21#issuecomment-998043321, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACR5T34NUNQUER5BLDFUZTUR5FYXANCNFSM5JI7GL3A . You are receiving this because you authored the thread.Message ID: @.***>