issues
search
IINemo
/
lm-polygraph
MIT License
111
stars
21
forks
source link
Fix/registers
#210
Closed
rvashurin
closed
1 month ago
rvashurin
commented
2 months ago
This PR does the following:
Implements new class - StatResolver, where
register_stat_calculators
function now lives.
Moves most of the stat calculator resolution from manager to this new class
Moves stats and dependencies of calculators to class-level function
meta_info
StatResolver registers all StatCalculators without instantiating
Then inits only stat calculators that are needed for this particular run
NLI model and OpenAIChat are only created once there is need to init calculator that requires it
Stat calculator resolution now triggered only when Manager starts running (
call
) to make loading saved managers faster
Some refactoring here and there
This PR does the following:
register_stat_calculators
function now lives.meta_info