ApolloResearch / rib

Library for methods related to the Local Interaction Basis (LIB)

MIT License

3 stars 0 forks source link

Support jacobian basis (formerly November B): Refactor version. #294

Closed stefan-apollo closed 6 months ago

stefan-apollo commented 6 months ago

Jacobian basis

Description

Support jacobian basis (formerly November B). See #244 for plots etc.

Related Issue

Replaces #244 due to refactor.

How Has This Been Tested?

Graph build tests, incl very small Pythia
Compared a few results (see below) to #244
Ideally but unrealistic: Add a test for "stray edges" now being gone.
- Not adding this test as this PR would not completely fix the issue, and it's likely another floating point but. See issue #276 where that is tracked,

Does this PR introduce a breaking change?

No

stefan-apollo commented 6 months ago

Results match those from previous branch for transformers.

Graph comparison:

New branch:

Old branch:

Ablations

Compare to here: