Previously, running sweep with datasets with config names, such as super_glue:boolq, would error as Sweep would check for datasets existing without partitioning for configs as we do for elicit, which this now adds support for
Reproducing ex elk sweep --models bigscience/bloomz-7b1-mt --datasets super_glue:boolq
Previously, running
sweepwith datasets with config names, such assuper_glue:boolq, would error as Sweep would check for datasets existing without partitioning for configs as we do forelicit, which this now adds support forReproducing ex
elk sweep --models bigscience/bloomz-7b1-mt --datasets super_glue:boolq