When fine-tuning and using `query_instruction_for_retrieval` should document instruction be blank?

FlagOpen / FlagEmbedding

Retrieval and Retrieval-augmented LLMs

MIT License

6.99k stars 510 forks source link

When fine-tuning and using `query_instruction_for_retrieval` should document instruction be blank? #387

Open austinmw opened 8 months ago

austinmw commented 8 months ago

If I followed the fine-tuning instructions and added a query_instruction_for_retrieval, should I use the same, different, or blank ("") instruction for the document ingestion part?

staoxiao commented 8 months ago

The query_instruction_for_retrieval will be added to query during fine-tuning. If you set this hyper-parameter, you should add it to query during inferencing. There is no need to add this instruction to document.