Open jeremierostan opened 1 week ago
Hi! I think you need to explicitly pass question, i.e.
qa = RAG()
answer = qa(question=“xxx?”).answer
Does this work for your case?
The RAG itself works:
qa = RAG() pred = qa("What is an AI dashboard?") print(f"Predicted Answer: {pred.answer}")
What does not work is compiling the module:
from dspy.teleprompt import BootstrapFewShot def validate_context_and_answer(example, pred, trace=None): answer_EM = dspy.evaluate.answer_exact_match(example, pred) answer_PM = dspy.evaluate.answer_passage_match(example, pred) return answer_EM and answer_PM teleprompter = BootstrapFewShot(metric=validate_context_and_answer) compiled_rag = teleprompter.compile(RAG(), trainset=trainset)
Any idea why? I am using the signature and module from the documentation (https://dspy-docs.vercel.app/docs/tutorials/rag). What I changed is using my own article a dataset:
from dspy.datasets import DataLoader dl = DataLoader() qa = dl.from_csv("/content/ai_integration_qa.csv") splits = dl.train_test_split(qa, train_size=15) train_dataset = splits['train'] test_dataset = splits['test'] trainset = [x.with_inputs('question') for x in train_dataset] devset = [x.with_inputs('question') for x in test_dataset]
I can't figure out what the issue is...
Hey @jeremierostan ! Do pass keyword arguments, like @mlederbauer said.
qa = RAG() pred = qa("What is an AI dashboard?") print(f"Predicted Answer: {pred.answer}")
Hi, yes, I have it here and it was already working:
qa = RAG() pred = qa("What is an AI dashboard?") print(f"Predicted Answer: {pred.answer}")
What is not working is the compiling:
from dspy.teleprompt import BootstrapFewShot def validate_context_and_answer(example, pred, trace=None): answer_EM = dspy.evaluate.answer_exact_match(example, pred) answer_PM = dspy.evaluate.answer_passage_match(example, pred) return answer_EM and answer_PM teleprompter = BootstrapFewShot(metric=validate_context_and_answer) compiled_rag = teleprompter.compile(RAG(), trainset=trainset)
Passing keyword arguments means:
Do not write pred = qa("What is an AI dashboard?")
Instead, write pred = qa(question="What is an AI dashboard?")
Apply this in every call to a module. I hope this helps.
Hi,
qa = RAG() pred = qa("xxx?") print(f"Predicted Answer: {pred.answer}")
works with the following, but compiling does not. Can anyone help?
from dspy.datasets import DataLoader dl = DataLoader() JR_qa = dl.from_csv("/content/ai_integration_qa_no_headers.csv") splits = dl.train_test_split(JR_qa, train_size=15) #
dataset` is a List of dspy.Example train_dataset = splits['train'] test_dataset = splits['test']trainset = [x.with_inputs('question') for x in train_dataset] devset = [x.with_inputs('question') for x in test_dataset]
class GenerateAnswer(dspy.Signature): """Answer questions with short factoid answers."""
class RAG(dspy.Module): def init(self, num_passages=3): super().init()
from dspy.teleprompt import BootstrapFewShot
Validation logic: check that the predicted answer is correct.
Also check that the retrieved context does actually contain that answer.
def validate_context_and_answer(example, pred, trace=None): answer_EM = dspy.evaluate.answer_exact_match(example, pred) answer_PM = dspy.evaluate.answer_passage_match(example, pred) return answer_EM and answer_PM
Set up a basic teleprompter, which will compile our RAG program.
teleprompter = BootstrapFewShot(metric=validate_context_and_answer)
Compile!
compiled_rag = teleprompter.compile(RAG(), trainset=trainset) `
——>
TypeError Traceback (most recent call last) in <cell line: 14>()
12
13 # Compile!
---> 14 compiled_rag = teleprompter.compile(RAG(), trainset=trainset)
4 frames
/usr/local/lib/python3.10/dist-packages/dspy/primitives/program.py in call(self, *args, kwargs) 24 25 def call(self, *args, *kwargs): ---> 26 return self.forward(args, kwargs) 27 28 def named_predictors(self):
TypeError: RAG.forward() missing 1 required positional argument: 'question'