search

adamkarvonen / SAEBench

7 stars 10 forks source link

model_name arg in core seems to do nothing #38

Open hijohnnylin opened 4 days ago

hijohnnylin commented 4 days ago

model_name arg in core seems to do nothing?

adamkarvonen commented 4 days ago

It looks like it is instead pulling model_name from the sae config. https://github.com/adamkarvonen/SAEBench/blob/main/evals/core/main.py#L956

So, it's fine for now, but this is confusing and should be fixed. I will wait until core eval sweeps are done to fix this.