Introduce dict[str, Performance]

neulab / ExplainaBoard

Interpretable Evaluation for AI Systems

MIT License

361 stars 36 forks source link

Introduce dict[str, Performance] #511

Closed odashi closed 2 years ago

odashi commented 2 years ago

Related to #491

This PR introduces following changes:

Remove metric_name member from Performance.
Change all list[Performance] to dict[str, Performance], whose keys are metric names.
Refactoring related to these changes.

Originally, metric_name is used as a primary key of the list, and often the list is converted to dict in the library. This change attempts to removes this burden.

pfliu-nlp commented 2 years ago

One high-level comment that could be considered as future PRs.

refactoring metric_configs, overall and analyses to dict[str, ...] as well

odashi commented 2 years ago

One high-level comment that could be considered as future PRs. refactoring metric_configs, overall and analyses to dict[str, ...] as well

Yes, I am planning to do it. The reason why this PR is proposed first is Performance is the most outer data structure that relies on metric names, and changing any inner structures affects this struct. I will propose other pull requests as well in the outer-inner manner.