search

fynnfluegge / codeqai

Local first semantic code search and chat powered by vector embeddings and LLMs
Apache License 2.0
385 stars 46 forks source link

Support for reStructuredText, and Markdown #5

Closed jasgeo75 closed 11 months ago

jasgeo75 commented 11 months ago

Treesitter was a brilliant addition. Any chance we might see support added for reStructuredText (rst) and Markdown (md) files?

fynnfluegge commented 11 months ago

Treesitter was a brilliant addition. Any chance we might see support added for reStructuredText (rst) and Markdown (md) files?

Sure, will add C#, Cpp and C support very soon. Then rst, md and yaml with its treesitter grammars. 🙌

Feel free to raise PR if you are faster haha 😄

fynnfluegge commented 11 months ago

@jasgeo75 any suggestion how to incorporate treesitter with markdown or rst? I have the feeling textsplitters by langchain are sufficient here, wdyt?

jasgeo75 commented 11 months ago

@fynnfluegge The reStructuredText and Markdown parsers for Treesitter appear to be for Rust. Langchain textsplitters are probably the way to go.