search

tianyi-lab / Cherry_LLM

[NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other models
306 stars 21 forks source link

The ifd score is affected by the prompt #22

Closed lihongxiacream closed 2 months ago

lihongxiacream commented 2 months ago

If I modify the template, all the calculated IFD values will be greater than 1 From this "prompt_input": ( "Below is an instruction that describes a task, paired with an input that provides further context. " "Write a response that appropriately completes the request.\n\n" "### Instruction:\n{instruction}\n\n### Input:\n{input}\n\n### Response:" ), to "prompt_input": ( "Below is an instruction that describes a task, paired with an input that provides further context. " "Write a response that appropriately completes the request.\n\n" "<|im_start|>system\n{instruction}<|im_end|>\n<|im_start|>user\n{input}<|im_end|>\n<|im_start|>assistant\n" )

MingLiiii commented 2 months ago

Thank you for your interest in our work! When you use a template that is unfamiliar to the base model, the IFD scores will increase as the overall formatted instructions are hard for the base model to understand.

Thus:

  1. You need to train a pre-experienced model, which is what exactly Cherry LLM does.
  2. Or you can directly remove the templates, which is what Superfiltering does. Please feel free to take a look at Superfiltering's implementation.