search

declare-lab / instruct-eval

This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.
https://declare-lab.github.io/instruct-eval/
Apache License 2.0
528 stars 42 forks source link

Support for larger batch_size #18

Closed soumyasanyal closed 1 year ago

soumyasanyal commented 1 year ago

Hi,

Is there any plan to extend this to run with batch_size greater than 1? Looking through the codebase it seems the general assumption is that batch_size has to be 1.

chiayewken commented 1 year ago

Hi, for now we do not plan to change the batch size to keep the repo simple