Fix tokenizer kwarg in unifiedTransformer

ndif-team / nnsight

The nnsight package enables interpreting and manipulating the internals of deep learned models.

https://nnsight.net/

MIT License

356 stars 34 forks source link

Open Butanium opened 3 weeks ago

Butanium commented 3 weeks ago

fixes #202

Butanium commented 3 weeks ago

Might be a good time to also fix https://github.com/ndif-team/nnsight/issues/138 by enforcing padding_side="left" for unified_transformer too ?

JadenFiotto-Kaufman commented 2 weeks ago

@Butanium Should we just get rid of enforcing padding side all together?

Butanium commented 2 weeks ago

@JadenFiotto-Kaufman I think we can probably do it but we'd need to ensure that the token indexing actually works in both cases