Closed assaftibm closed 4 days ago
Suggested fix from @yoavkatz :
from unitxt.collections_operators import Wrap
metric = MetricPipeline(
main_score=“score”,
preprocess_steps=[
Copy(field=“context_ids”, to_field=“prediction”),
Wrap(field=“ground_truths_context_ids”, inside=“list”, to_field=“references”),
],
metric=f”metrics.{metric_name}“,
)
The underlying implementation has changed, and now it requires the reference to be provided as List[List[str]] rather than just List[str]. We want to expose the metric in its standard definition, where the reference is List[str]. @benjaminsznajder noticed this and can give more details.