IMPORTANT API CHANGE: Metrics Re-Organization

dilyabareeva / quanda

A toolkit for quantitative evaluation of data attribution methods.

https://quanda.readthedocs.io

MIT License

33 stars 0 forks source link

IMPORTANT API CHANGE: Metrics Re-Organization #134

Closed dilyabareeva closed 2 months ago

dilyabareeva commented 2 months ago

Since we don't have enough metrics to fill the original categories (faithfulness, localization, etc.), for now I introduce the following simple and in my opinion intuitive separation, also reflected in the text of the paper:

heuristics
downstream task evaluators
ground truth

PLUS:

add tests for sample.py dataset and random.py explainer
remove unused files (felt they were useless, might revert later)

dilyabareeva commented 2 months ago

@aski02 @gumityolcu I hope you see this 😄