Adds unit tests for all_loadable_saes() and get_saes_from_regex(), and another test for run_evals(). There's no coverage for the first two functions, and there's no coverage for single_head_replacement_hook() and single_head_zero_ablate_hook() defined inside of the third function, at https://app.codecov.io/gh/jbloomAus/SAELens/blob/main/sae_lens%2Fevals.py.
Fixes #82
Type of change
Please delete options that are not relevant.
[x] Bug fix (non-breaking change which fixes an issue)
Checklist:
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I have not rewritten tests relating to key interfaces which would affect backward compatibility
You have tested formatting, typing and unit tests (acceptance tests not currently in use)
[x] I have run make check-ci to check format and linting. (you can run make format to format code if needed.)
Performance Check.
If you have implemented a training change, please indicate precisely how performance changes with respect to the following metrics:
[ ] L0
[ ] CE Loss
[ ] MSE Loss
[ ] Feature Dashboard Interpretability
Please links to wandb dashboards with a control and test group.
Description
Adds unit tests for
all_loadable_saes()andget_saes_from_regex(), and another test forrun_evals(). There's no coverage for the first two functions, and there's no coverage forsingle_head_replacement_hook()andsingle_head_zero_ablate_hook()defined inside of the third function, at https://app.codecov.io/gh/jbloomAus/SAELens/blob/main/sae_lens%2Fevals.py.Fixes #82
Type of change
Please delete options that are not relevant.
Checklist:
You have tested formatting, typing and unit tests (acceptance tests not currently in use)
make check-cito check format and linting. (you can runmake formatto format code if needed.)Performance Check.
If you have implemented a training change, please indicate precisely how performance changes with respect to the following metrics:
Please links to wandb dashboards with a control and test group.