parthsarthi03 / raptor

The official implementation of RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
https://arxiv.org/abs/2401.18059
MIT License
687 stars 98 forks source link

RAG seems lazy and don't understand the sample #7

Open thhung opened 4 months ago

thhung commented 4 months ago

Hello, I used the demo notebook you provided with some questions. Most of the time the model cannot answer the questions that we can find the answers from the sample's text. Could you provide some guidance to overcome this issue?

Example:

Question: "How did her life became hard?" 
Answer:  Sure, here's the answer to your question: The passage does not provide any information about how her life became hard, so I cannot answer this question from the context.

Maybe something wrong with the context? The code is exactly the same as your demo, except the few code added to install dependencies and correct typos.

parthsarthi03 commented 4 months ago

I'm not able to reproduce this: in the demo when I pass in your question, I get the following answer with the default settings.

Question:  How did her life became hard?
Answer:  Her life became hard after her mother passed away and her father remarried a woman with two cruel daughters. The stepmother and stepsisters mistreated Cinderella, making her do all the household chores and mocking her appearance. She was forced to sleep by the hearth among the cinders, given menial tasks, and subjected to ridicule and sabotage by her stepsisters. Despite her pious and good nature, Cinderella faced constant hardship and cruelty from her new family, leading to a difficult and challenging life.

Are you using a custom model for QA/Summarization or the default? What are the extra dependencies/typos?