search

Upaya07 / NeurIPS-llm-efficiency-challenge

Code for NeurIPS LLM Efficiency Challenge
Apache License 2.0
54 stars 9 forks source link

What is the role of Mistral Inference during Natural-Instruction dataset preparation? #1

Open gauss5930 opened 11 months ago

gauss5930 commented 11 months ago

First, I would like to congratulate you on winning first place in the NeurIPS2023 llm-efficiency challenge! 🥳🥳

I wrote an issue since I've got a question while reading Repo's README introduced Birbal. I understand that in the process of collecting training data, the Natural-Instruction dataset goes through 'relevant task selection' and inference is performed with the Mistral model. My question is why inference was performed? I could not fully understand by just referring README file 😭

I would appreciate it if you let me know!

akjindal53244 commented 11 months ago

Hi @gauss5930 , inference is performed to compare model's prediction with ground truth to decide whether a record should be sampled or not.