Closed idan-tankel closed 9 months ago
tried a few system prompts that might help. Some of them are using 4-choice question to 4-choice statements raking problem, and the other ones are converting pairs (Q-choice) into a single statement, one by one.
The question to ranking question conversion might work here as well.
another thing to consider here is that the perplexity test is using the FLAN-T5 module for generation (as part of the BLIP2). Is it possible to make the Statements "FLAN-T5 friendly"? to keep them somehow as far as possible in the FLAN-T5 embedding space? How the space of possible LLAMA2 generated answers looks under FLAN-T5 embeddings? Can we use FLAN-T5 for the rephrasing stage?
working. have the systemprompt commited
In text - Enhancement task of converting the question and answers into statements,
system_prompts_conversation
"System prompts" in question generation for the GPT
As written in SeedBench paper here, the questions were generated using the following prompt default instruction, Fig 4
This is very similar to the concept of system prompts