Metrics & Explainers: models and checkpoints are separate non-optional arguments

dilyabareeva / quanda

A toolkit for quantitative evaluation of data attribution methods.

https://quanda.readthedocs.io

MIT License

33 stars 0 forks source link

Open dilyabareeva opened 2 months ago

dilyabareeva commented 2 months ago

Why?

Model randomisation: we need to randomize all the checkpoints
Some explainers require checkpoints -> we need to save checkpoints in the training process in benchmarks, and checkpoints should probably become a part of base benchmark signature