Closed xxm1668 closed 7 months ago
We have merged the DoLa decoding into the new version (4.39.0.dev0) of transformers package. Install it here: https://github.com/voidism/transformers-dola Follow the instructions here for decoding: https://github.com/voidism/transformers-dola/blob/main/docs/source/en/generation_strategies.md#dola-decoding This implementation is agnostic to the model architecture, so it should support new models in huggingface.
4.39.0.dev0
We have merged the DoLa decoding into the new version (
4.39.0.dev0) of transformers package. Install it here: https://github.com/voidism/transformers-dola Follow the instructions here for decoding: https://github.com/voidism/transformers-dola/blob/main/docs/source/en/generation_strategies.md#dola-decoding This implementation is agnostic to the model architecture, so it should support new models in huggingface.