LAION-AI / Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
https://open-assistant.io
Apache License 2.0
37.04k stars 3.23k forks source link

Fix Rope scaling #3598

Closed shahules786 closed 1 year ago

shahules786 commented 1 year ago

What

Fixed rope scaling for all models.

Why

Earlier the model config was being ignored during patching which might cause issues like initializing with wrong max_position_embeddings

How

Gets required args from model_config and passes it to patching functions.