search

ApolloResearch / rib

Library for methods related to the Local Interaction Basis (LIB)
MIT License
3 stars 0 forks source link

Read nodes_per_layer from file in plot_mlp_graph #236

Closed stefan-apollo closed 7 months ago

stefan-apollo commented 9 months ago

plot_mlp_graph.py:

    nodes_per_layer = [40, 10, 10, 10]

is hardcoded but should adjust to the number of layers in the input file

Also let's not force overwrite by default:

force: bool = True
nix-apollo commented 9 months ago

I'm not sure that we want these to be the same as the hidden sizes of the mlp itself? I think [40, 10, 10, 10] is a fine default for MNIST mlps, for instance. Do you have suggestions for how to pick a better default?

stefan-apollo commented 9 months ago

Oh the sizes are fine but the number of layers can vary -- I was confused why the script crashed for my 5-layer DNN until I realized that

danbraunai-apollo commented 8 months ago

This will get squashed when I do the refactor that merges MLP and LM scripts. I guess I'll set the default to ~15 nodes in all layers.

danbraunai-apollo commented 8 months ago

In https://github.com/ApolloResearch/rib/pull/260, we simply default to 40 nodes per layer, but allow specifying an int or list of ints for how many nodes per layer (if int, then apply the same number of nodes to all layers).

I think this is fine for now, probably not worth customising more than this unless someone wants to.