search

HuskyInSalt / CRAG

Corrective Retrieval Augmented Generation
277 stars 26 forks source link

About Self-CRAG. #13

Open edwarddiou opened 5 months ago

edwarddiou commented 5 months ago

Thanks for the great job ! I'd like to ask how the extracted "ambiguous," "correct," and "incorrect" parts are used in Self-RAG after being identified in Self-CRAG. Do it only keep the correct portions?

Another quick question, if I directly set the --generator_path as selfrag/selfrag_llama2_7b in CRAG_Inference.py, will the results be different from the ones above?

Thank you.

HuskyInSalt commented 3 months ago

Hi @edwarddiou ! In CRAG, we introduced an extra evaluator to evaluate the reliability of the retrieved context and three actions are triggered, resulting in three different types of input for the generator. However in Self-CRAG, as we mentioned in the paper, considering that Self-RAG can evaluate the quality of the retrieved context and decide which context to be used for generation, we simply provide all three types of input ( "ambiguous," "correct," and "incorrect") to Self-RAG and let itself decide which action to be triggered.

If you directly set the --generator_path as selfrag/selfrag_llama2_7b, you just change a different generator, but the framework (CRAG) does not change and the evaluator is still required.

edwarddiou commented 2 months ago

Thank for your response.

May I ask why you chose to use the T5 model instead of the BERT for the CRAG pre-trained models?