search

AlignmentResearch / tuned-lens

Tools for understanding how transformer predictions are built layer-by-layer
https://tuned-lens.readthedocs.io/en/latest/
MIT License
432 stars 47 forks source link

Support Mistral #133

Open norabelrose opened 6 months ago

norabelrose commented 6 months ago

Adds MistralModel to the model surgery functions

norabelrose commented 6 months ago

Some very strange test failure is happening involving datapipes

levmckinney commented 5 months ago

Looks like their is an issue with pytorch 2.3 that is causing it to be incompatible with torchdata https://github.com/pytorch/pytorch/pull/126094.

It should be fixed in pytorch 2.3.1

levmckinney commented 5 months ago

@norabelrose rebase or merge this with main and it should work.