search

victorlei / smop

Small Matlab to Python compiler
MIT License
1.08k stars 410 forks source link

New Parser and Mathworks official Mathworks grammar #144

Closed RobBW closed 5 years ago

RobBW commented 5 years ago

  1. Github have adopted a new parser technology for Atom called Tree-Sitter which improves Matlab parsing. See: https://tree-sitter.github.io/tree-sitter/
    A descriptive video is at: https://www.thestrangeloop.com/2018/tree-sitter---a-new-parsing-system-for-programming-tools.html Tree-Sitter looks as if it may deal effectively with some of the trickier parsing problems that you encounter with Matlab.

  2. One of the things that has hindered development of transpilers for Matlab was lack of an official grammar. This has been resolved recently and as of 4 months ago an official grammar is available at: https://github.com/mathworks/MATLAB-Language-grammar/tree/master/Matlab.tmbundle/Syntaxes ()

  3. This new grammar is now used in the Atom code editor with their new parser generator tool Tree-Sitter. It greatly improves Atom's syntax colouring decisions. It may do the same for smop.

You may wish to investigate.

makslevental commented 5 years ago

@RobBW which matlab plugin for atom uses tree sitter and the new grammar? i can't find it

RobBW commented 5 years ago

Tree sitter is now the primary parser within Atom. I think I used the language-matlab package in Atom. I cannot verify this because I am away from my home computer.

In case it helps here are my Notes about Treesitter:

Tree-sitter is a parser generator tool and an incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited. It is now integrated into Atom. https://tree-sitter.github.io/tree-sitter/

Creating parsers: https://tree-sitter.github.io/tree-sitter/creating-parsers

Creating a Grammar: https://flight-manual.atom.io/hacking-atom/sections/creating-a-grammar/

Converting a Grammar: from TextMate: https://flight-manual.atom.io/hacking-atom/sections/converting-from-textmate/

There is a TextMate version of the Matlab Official Grammar from Mathworks (most recent 2019) at NEW:

https://github.com/mathworks/MATLAB-Language-grammar/tree/master/Matlab.tmbundle/Syntaxes

****This appears to have been converted and is used in the …. Atom Matlab package.

It is more up to date than the Textmate site: https://github.com/textmate/matlab.tmbundle/tree/master/Syntaxes

Regards Rob

On 21/08/2019, at 09:16, Maksim Levental notifications@github.com wrote:

@RobBW which matlab plugin for atom uses tree sitter and the new grammar? i can't find it

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

makslevental commented 5 years ago

@RobBW thanks. the issue with your idea is that the grammar released by matlab isn't an LLk grammar - just a regex grammar - so you almost certainly can't use it for transpilation.

RobBW commented 5 years ago

I’m now well out of my depth in understanding the complexities of different grammars. I think the information was worth documenting though. I’ll close the issue now.