Train Pre-Experienced Model

[Schopenhauer-loves-Hegel]

So this method ignore Train Pre-Experienced Model step mentioned in the previous paper From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning. In the previous paper, the pre-finetuning step is deemed rather important. May I inquire if this step was removed from the repository and the paper due to its negligible impact?

[MingLiiii]

Thank you so much for your interest in our work!

Direct answer of your question: The pre-finetuning step is kept in the codes and paper of the Cherry_LLM. It is removed in the codes and paper in the of the Superfiltering.

Main Differences: In the paper From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning, we calculate the conditioned Loss/PPL in IFD scores with the instruction-response template, thus LLMs need to be equipped with the basic instruction-following ability to get used to the template. However, in the paper Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning, we optimized this process by discarding the template when calculating the conditioned Loss/PPL, which is important for the consistency between small and large language models. This difference is directly shown in the code.

We will add this to the main pages of our repos in the next version.

[Schopenhauer-loves-Hegel]

Thank you for your reply! I would like to clarify my understanding.

If I have a raw dataset that consists of prompt-response pairs and I don't want to use any templates, can I simply use the superfilter method to assess the data quality without going through the step of training a pre-experienced model?

[MingLiiii]

Yes, you are correct! That is exactly why we proposed Superfiltering, which is much simpler and faster.