Support for gemma architecture models

AlignmentResearch / tuned-lens

Tools for understanding how transformer predictions are built layer-by-layer

https://tuned-lens.readthedocs.io/en/latest/

MIT License

437 stars 47 forks source link

Support for gemma architecture models #125

Closed levmckinney closed 5 months ago

levmckinney commented 9 months ago

We also need to bump some dependencies for this since gemma does not exist in the version of transformers we were requiring.

norabelrose commented 9 months ago

Sounds good, it looks like it's currently failing because the mockmodel doesn't exist on the HF hub or something? I'm not super familiar with your unit tests, I'd need to look at it more carefully.

I'm definitely fine with bumping the transformer dependency, we could also drop Python 3.9 support too while we're at it (enabling | for union types)

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

:loudspeaker: Thoughts on this report? Let us know!

norabelrose commented 6 months ago

Looks like we're just failing on TypeError: Subscripted generics cannot be used with class and instance checks, shouldn't be hard to fix