search

neulab / ExplainaBoard

Interpretable Evaluation for AI Systems
MIT License
361 stars 36 forks source link

Fix Processor._customize_analyses #581

Closed odashi closed 1 year ago

odashi commented 1 year ago

Overview

This PR restores the previous behavior in Processor._customize_analyses.

Details

Processor._customize_analyses was changed resently to "add" given metrics to default metrics, but the original behavior was "replace" it with the given metrics. This change restores the original behavior. This change also introduces MetricConfig.replace_languages and AnalysisLevel.replace_metric_configs functions, which creates a new object by copying self with replacing several members.

References

Blocked by

neubig commented 1 year ago

Looks like there is a type-related error on Python 3.8, but I can take a look after that.

odashi commented 1 year ago

@neubig It was resolved.