ApolloResearch / rib

Library for methods related to the Local Interaction Basis (LIB)

MIT License

2 stars 0 forks source link

WIP: Analytic integrated gradients linear layer #210

Closed danbraunai-apollo closed 11 months ago

danbraunai-apollo commented 11 months ago

NOTE: POSTPONED AFTER STOCHASTIC SOURCES DISCOVERY. Analytic integrated gradients for a plain linear layer

Description

When calculating edges, checks if a section contains mlp_in or mlp_out and uses the analytical integrated gradient formula instead of numerical

Related Issue

Closes #206

Motivation and Context

How Has This Been Tested?

TODO: Compare run with numerical integrated gradients (previous commit) to this run. They should be the same.

Does this PR introduce a breaking change?