Oh other totally random idea that I had yesterday at an AI + Science talk (on equivariant neural networks — very interesting):
One of the weird things with the ROME edit is it doesn't change the relationship for the "target" if that makes sense. So editing "The Space Needle is in Chicago" doesn't change the output for "Chicago has many great landmarks like..."
But it seems like if the input is equivariant it should actually do that.
Additionally, sample efficiency for equivariant networks is way higher than just invariant networks and generalizability is way better, but at a much higher computation cost.
Seems plausible that humans encode information and conceptual relationships in an equivariant format
Oh other totally random idea that I had yesterday at an AI + Science talk (on equivariant neural networks — very interesting): One of the weird things with the ROME edit is it doesn't change the relationship for the "target" if that makes sense. So editing "The Space Needle is in Chicago" doesn't change the output for "Chicago has many great landmarks like..." But it seems like if the input is equivariant it should actually do that. Additionally, sample efficiency for equivariant networks is way higher than just invariant networks and generalizability is way better, but at a much higher computation cost. Seems plausible that humans encode information and conceptual relationships in an equivariant format