Closed gkielian closed 2 months ago
This will allow not setting the abs pos embedding table if one wants to instead finetune Rotary, functional embeddings, or other alternatives which allow for better context length extensions.
To be merged in after PR #262
This will allow not setting the abs pos embedding table if one wants to instead finetune Rotary, functional embeddings, or other alternatives which allow for better context length extensions.
To be merged in after PR #262