TransformerLensOrg / TransformerLens

A library for mechanistic interpretability of GPT-style language models

https://transformerlensorg.github.io/TransformerLens/

MIT License

1.45k stars 283 forks source link

Add tests for gated mlp #638

Closed anthonyduong9 closed 3 months ago

anthonyduong9 commented 3 months ago

Description

Adds tests for gated_mlp.py. When you sort on test coverage, this file has the worst.

Type of change

Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist:

[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I have not rewritten tests relating to key interfaces which would affect backward compatibility

bryce13950 commented 3 months ago

Perfect! Just one minor change with where the test should be, but I took care of that. Keep it in mind if you add more of these tests