motion-canvas / motion-canvas

Visualize Your Ideas With Code
https://motioncanvas.io
MIT License
16.1k stars 605 forks source link

Switch code node to use treesitter #1028

Closed IogaMaster closed 6 months ago

IogaMaster commented 6 months ago

Description Lezer is a quite obscure, with little language support (15 parsers). As an example nix, bash, and c# are not supported.

Proposed solution Use treesitter, https://github.com/tree-sitter/tree-sitter

Considered alternatives

Try to get a nix lezer parser. However this is not preffered.

Not Happening :cry:

IogaMaster commented 6 months ago

Nevermind, we arent getting a nix lezer parser.

image

IogaMaster commented 6 months ago

https://github.com/tree-sitter/node-tree-sitter Seems to almost match the api you use already.

aarthificial commented 6 months ago

Here's a really simple example of using the shell parser from @codemirror/legacy-modes (There's a few more parsers there than the 15 you've mentioned): image

Source code ```tsx import {StreamLanguage} from '@codemirror/language'; import {shell} from '@codemirror/legacy-modes/mode/shell'; import {Code, LezerHighlighter, makeScene2D} from '@motion-canvas/2d'; import {createRef, waitFor} from '@motion-canvas/core'; const Bash = new LezerHighlighter(StreamLanguage.define(shell).parser); export default makeScene2D(function* (view) { const code = createRef(); view.add( > ../node.log & # here is how to stop the server for pid in \`ps aux | grep 'node ../server.js' | awk '{print $2}'\` ; do sudo kill -9 $pid 2> /dev/null done exit 0 `} />, ); yield* waitFor(0.6); }); ```

https://github.com/tree-sitter/node-tree-sitter Seems to almost match the api you use already.

The rendering part of Motion Canvas doesn't run on Node.js. You'd need the WASM bindings. The API is similar because Lezer is based on Treesitter.


I'm closing this issue because I'm not interested in working on this in the context of this repository.

The CodeHighlighter interface is independent of Lezer and can be used to integrate any parser/highlighter solution into Motion Canvas without changing our code. When designing the interface, I made sure to account for Treesitter as that's what I used for prototyping.

Sadly, their documentation (both the wasm and node.js one) does not describe how to highlight the parsed tree so I gave up on the implementation and went for Lezer.

If anyone wants to give Treesitter a try, they can easily do that in their own npm package because, again, it's just a matter of implementing an interface - no changes to this repo necessary. Once a package like this exists I'll gladly talk about upstreaming it here.