search

yitzchak / tree-sitter-latex

LaTeX grammar for tree-sitter
MIT License
11 stars 5 forks source link

Category based display/inline math shift parsing #18

Closed yitzchak closed 5 years ago

yitzchak commented 5 years ago

This PR adds INLINE_MATH_SHIFT and DISPLAY_MATH_SHIFT tokens. This brings over some of the ideas from #17.

Aerijo commented 5 years ago

What does this do for foo{$}bar? That was the reason I made contents optional for inline math too, and I think what lead to the manual scanner rules.

Where possible, I want to avoid ERROR and MISSING nodes. It makes any tool that uses the tree have to work harder to see if it's an actual error or not. That's the same reason I used implicit_math_shift instead of the missing handler.

yitzchak commented 5 years ago

An error is currently produced for {$}. {foo$} produces MISSING.

Why do you want to avoid MISSING? Seems like having MISSING in the tree will still allow syntax coloring.

Also, under what context are you seeing things like {foo$bar} in a LaTeX document? Just curious why someone would want to have a dangling math shift in a text group.

Aerijo commented 5 years ago

Why do you want to avoid MISSING? Seems like having MISSING in the tree will still allow syntax coloring.

It wasn't about syntax highlighting, it was for other tools (such as a generic linter; I'm not sure how useful this is in practice, but it was simple to make).

Also, under what context are you seeing things like {foo$bar} in a LaTeX document

Nothing in particular, just a combination of managing language-latex issues (I can't find any, but I recall several are related to math detection) and accepting that TeX users often do whacky things (but not even that whacky). I don't think we have a chance of accounting for all valid LaTeX, but I do think we can do our best to accommodate.

I may be confusing it with the >{$} stuff for tables though, which I think is supported by the existing grammar.

Aerijo commented 5 years ago

@yitzchak I was going to attempt serialising, unless you've already started / want to do it?

yitzchak commented 5 years ago

@Aerijo Sounds good! Serialization is yours. Can you create the branch on this repo?

yitzchak commented 5 years ago

@Aerijo I'm going to merge this once I get a green board. I did not add the implicit math shift token, but I am getting basically the same behavior on the tests as #17. We can explore the implicit math shift idea in a separate PR if needed.