nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath
9.21k stars 715 forks source link

Question about Elimination Evolving #220

Open yfzhou3993 opened 10 months ago

yfzhou3993 commented 10 months ago

HI!

I've been very interested in your research and attempted to apply the process described in the WizardLM paper to evolve instructions on the Alpaca dataset. After one round of evolution, only roughly half of the new instructions passed when I used the first method of Elimination, i.e., determining if the evolved instructions were equal to the original ones as judged by GPT. Were the same results observed in your experiment?

In addition, Section 4.2 of the WizardLM paper mentions, "To construct the dataset, we initialized it with the 52K instruction dataset of Alpaca. After iteratively performing M evolutions, where M = 4, we obtained a 250K instructions." Are these 250K instructions already post the Elimination process?

I am seeking guidance as to whether there's any bias in my experimental results, or perhaps a misunderstanding on my part.

I appreciate your time in assisting me on this matter.