Currently marked and highlight.js are used to render markdown and highlight code, while they are great libraries and very good performance, they don't support streaming input. So currently when the streaming conversation is rendered the implementation is just re-rendering everything every time when a new character is received.
The tree-sitter is very good at this use case and we should switch to it for parsing.
Currently
markedandhighlight.jsare used to render markdown and highlight code, while they are great libraries and very good performance, they don't support streaming input. So currently when the streaming conversation is rendered the implementation is just re-rendering everything every time when a new character is received.The tree-sitter is very good at this use case and we should switch to it for parsing.