[Gen-Ai-Orchestrator] Add title to embedded documents for better retrieval

theopenconversationkit / tock

Tock, the open source conversational AI toolkit.

https://doc.tock.ai

Apache License 2.0

505 stars 134 forks source link

[Gen-Ai-Orchestrator] Add title to embedded documents for better retrieval #1639

Closed GuirriecP closed 2 months ago

GuirriecP commented 5 months ago

Adds the 'Title' entry of input CSV to embedded documents:

during retrieval, the list of retrieved documents is more deterministic
(seems to) slightly improves the quality of the generated answer
fix the 'Text: ' préfix present in first chunks of source documents.

Fixes #1638

vsct-jburet commented 5 months ago

@GuirriecP is it still draft ?

GuirriecP commented 5 months ago

@vsct-jburet Yes it is. We will draft PRs to tell the Arkéa team members that the work is done, pending review. We will only push final, mergeable PRs to the repo when they have been reviewed.

Benvii commented 2 months ago

@assouktim were did you fix this PR ? is there a new branch somewhere ? thanks in advance

assouktim commented 2 months ago

Replaced by https://github.com/theopenconversationkit/tock/pull/1732