Can Dify retrival structured dataset? for example, .csv? and answer questions from nature language.

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

https://dify.ai

Other

45.3k stars 6.36k forks source link

Can Dify retrival structured dataset? for example, .csv? and answer questions from nature language. #6140

Closed AmyHei closed 2 months ago

AmyHei commented 2 months ago

Self Checks

[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

I want to upload structured data and retrival, answer question based my data. for example, which customer is the most profit one?

2. Additional context or comments

No response

3. Can you help us with this feature?

[X] I am interested in contributing to this feature.

crazywoola commented 2 months ago

Consider a scenario where you have data in the following format:

1, a, b, c
2, d, e, f

If you upload a CSV file to an embedding service and split the lines by \n, you can indeed ask questions regarding the data.

However, for more complex situations, such as fetching multiple records from this file, you should use the traditional method of retrieving data from a well-structured API.

A common approach is to upload the CSV file to a database and then use a language model (LLM) to parse natural language queries into SQL, allowing you to retrieve data accordingly.

mboo2005 commented 2 months ago

@crazywoola Does Dify support filtering during vector search, similar to the syntax used in Weaviate as shown above? In certain scenarios, it is necessary to deterministically filter data, as relying solely on vector search is not sufficient.