Add rib.models.modular_mlp.create_modular_mlp function for creating an MLP instance with block diagonal weights. This is a replacement for the ModularMLP class.
Remove the MLPConfig property from the ModularMLPConfig.
Related Issue
Motivation and Context
Having a ModularMLP class which is actually just an MLP with different weights really smells. It also made things awkward when considering what the type of the model.cfg should be (e.g. should your do a super() and then overwrite the parent self.cfg with the child self.cfg? And how to avoid type errors if doing this?).
It also really smelt having a MLPConfig property of a ModularMLP config.
How Has This Been Tested?
Same tests pass
Does this PR introduce a breaking change?
Only if a user used the ModularMLP class directly. I don't think this was the case for us since ModularMLP was only merged recently.
Use raw MLP class for modular MLPs
Description
rib.models.modular_mlp.create_modular_mlpfunction for creating an MLP instance with block diagonal weights. This is a replacement for the ModularMLP class.Related Issue
Motivation and Context
Having a ModularMLP class which is actually just an MLP with different weights really smells. It also made things awkward when considering what the type of the model.cfg should be (e.g. should your do a super() and then overwrite the parent self.cfg with the child self.cfg? And how to avoid type errors if doing this?).
How Has This Been Tested?
Does this PR introduce a breaking change?