Closed danbraunai-apollo closed 6 months ago
I agree it would be nice to have at least some test ablation scripts run rib build before ablations. When doing so it would be great for some of them to use a centered rib build (this would increase test coverage in ablations.py)
Currently, our tests in test_ablations require the following process: For both mnist and modadd:
python rib_scripts/rib_build/run_rib_build.py rib_scripts/rib_build/<config.yaml>
It will be better if we instead:
It seems like we can do option 1 for the tests very easily, and even keep it as a fast-running test. But the downside is that the default ablation configs use the sample_graphs, so we probably want these to be up to date anyway.
Best path might be to implement 1 but also add a script which trains the models and stores them in sample_graphs. Unsure when we should run this script.