neulab / explainaboard_web

MIT License
8 stars 2 forks source link

Add metric descriptions to backend #512

Closed noelchen90 closed 1 year ago

noelchen90 commented 1 year ago

This PR adds the mapping of metric names --> descriptions to the backend. This aims to solve Issue #426 UI Optimization Plan where we can explain metrics to users.

Comments

In metric_descriptions.py, the description strings had to be broken down in the form of "xxx"\n"xxx" to pass the format checker (error falke8 E501 line too long)

Discussion

In the beginning, we considered to save a dataclass by referencing the documentation supported_metrics.md

class MetricDescription:
    id: int
    name: str
    description: str
    used_tasks: list[TaskType]
    ref_url: str=""

But considering the use case for this class is only to look up the descriptions, and not the other attributes, we figured we actually only needed a dictionary. All the other information (reference URL and used tasks) can be found in the documentation if users wanted to learn more.

Next Steps

I will have two separate PRs for adding these descriptions in (1) system submit drawer and (2) analysis page.

Thanks to @lyuyangh for all the suggestions and discussions!

noelchen90 commented 1 year ago

Sorry for the multiple formatting commits. I am facing the deadlock issue where my local black format check wants a set of import As after import Bs, while github's isort format check wants import As before import Bs.

The latter commits used the tag --no-verify to skip those checks, that's why I had to format according to the messages from CI.

lyuyangh commented 1 year ago

Could you also bump the API version in openapi.yaml?