ApolloResearch / rib

Library for methods related to the Local Interaction Basis (LIB)
MIT License
3 stars 0 forks source link

Support CIFAR model training #223

Closed nix-apollo closed 9 months ago

nix-apollo commented 9 months ago

CIFAR

Description

Gives the MNIST scripts greater flexibility so we can now train mlps on other datasets (for now only CIFAR 10, but would be trivial to support other options in torchvision datasets).

In order to facilitate this, we:

Motivation and Context

We want to be able to examine other mlp-only models. Cifar is a reasonable next step up from mnist, although it's not great (models don't perform that well, adding more parameters don't seem to help that much)

How Has This Been Tested?

Updated old tests. I've also trained cifar models which get the expected performance. I haven't added tests for cifar specifically, not sure if this would be helpful.

Does this PR introduce a breaking change?

nix-apollo commented 9 months ago

Questions for @danbraunai-apollo :

  1. The ablation scripts are very similar. Maybe they should be combined. It would also be totally possible (if slightly more work) to combine the rib build scripts. I would be in favor of combining both if it was trivial, but leaning away just because spending the time refactoring is probably not worth it. Thoughts?
  2. Do we rename experiments/train_mnist to experiments/train_mlp (or _dnn)? Would also rename other folders and files for consistency.
nix-apollo commented 9 months ago

Due to the config change I trained another mnist model, although I'm realising this was silly and I could have easily manually edited the config. Perhaps I should revert that change and just adjust the config.

danbraunai-apollo commented 9 months ago
  1. The ablation scripts are very similar. Maybe they should be combined. It would also be totally possible (if slightly more work) to combine the rib build scripts. I would be in favor of combining both if it was trivial, but leaning away just because spending the time refactoring is probably not worth it. Thoughts?
  2. Do we rename experiments/train_mnist to experiments/train_mlp (or _dnn)? Would also rename other folders and files for consistency.
  1. I would like this. Not high priority. Just created an issue for it and assigned it as medium priority.
  2. train_mlp would be great. I strongly dislike "_dnn", because it might not be deep.