Closed stefan-apollo closed 7 months ago
I'm not sure that we want these to be the same as the hidden sizes of the mlp itself? I think [40, 10, 10, 10] is a fine default for MNIST mlps, for instance. Do you have suggestions for how to pick a better default?
Oh the sizes are fine but the number of layers can vary -- I was confused why the script crashed for my 5-layer DNN until I realized that
This will get squashed when I do the refactor that merges MLP and LM scripts. I guess I'll set the default to ~15 nodes in all layers.
In https://github.com/ApolloResearch/rib/pull/260, we simply default to 40 nodes per layer, but allow specifying an int or list of ints for how many nodes per layer (if int, then apply the same number of nodes to all layers).
I think this is fine for now, probably not worth customising more than this unless someone wants to.
plot_mlp_graph.py
:is hardcoded but should adjust to the number of layers in the input file
Also let's not force overwrite by default: