search

microsoft / TaskWeaver

A code-first agent framework for seamlessly planning and executing data analytics tasks.
https://microsoft.github.io/TaskWeaver/
MIT License
5.29k stars 667 forks source link

Can TaskWeaver do the indexing part (loading, splitting, storing)? #118

Closed ligjn closed 9 months ago

ligjn commented 9 months ago

Is your feature request related to a problem? Please describe. I want to use the document Q&A feature of TaskWeaver. Can TaskWeaver do the indexing part (loading, splitting, storing)?

Describe alternatives you've considered Indexing documents first like LangChain (loading, splitting, storing)，Or can I only use TaskWeaver to implement plugins for Retrieval and generation?

ShilinHe commented 9 months ago

Hi @ligjn, Thanks for your question! Currently, indexing is not included in TaskWeaver since this part is relatively independent and does not rely on human interaction. We would recommend first build an indexing application (many existing work) and incorporate the document retrieval and generation with TaskWeaver as a plugin.

ligjn commented 9 months ago

Okay, I understand