search

AlibabaResearch / DAMO-ConvAI

DAMO-ConvAI: The official repository which contains the codebase for Alibaba DAMO Conversational AI.
MIT License
1.1k stars 178 forks source link

how to select example in deep-thinking #75

Closed noobimp closed 10 months ago

noobimp commented 10 months ago

hi, I noticed that you chose "C ∗ N demonstrations for a task with C classes", so how to select them? I am not sure if I missed this description in the paper?

Yangjiaxi commented 10 months ago

Hi, @noobimp, we are delighted to hear that you're interested in our work! We select demonstration examples through random selection.

Specifically:

  1. For classification tasks, we use stratified sampling. In this method, we randomly sample $N$ examples from each class.
  2. For multiple-choice tasks, since there are no class labels in this setting, we simply sample $N$ examples from the entire set.

Whether we sample from the training set or validation set depends on their availability, and we guarantee that the test set is excluded.

You can find the implementation details here.