Hi,
The FewShotSampler samples 'num_examples_total' or 'num_examples_per_label' for the dev set even if setting different values for 'num_examples_total_dev' or 'num_examples_per_label_dev'. It only works as wished if valid_dataset=None and also_sample_dev=True.
yulinchen99
commented
1 year ago
Hi, The FewShotSampler samples 'num_examples_total' or 'num_examples_per_label' for the dev set even if setting different values for 'num_examples_total_dev' or 'num_examples_per_label_dev'. It only works as wished if valid_dataset=None and also_sample_dev=True.