Closed shahules786 closed 1 year ago
Fixed rope scaling for all models.
Earlier the model config was being ignored during patching which might cause issues like initializing with wrong max_position_embeddings
max_position_embeddings
Gets required args from model_config and passes it to patching functions.
What
Fixed rope scaling for all models.
Why
Earlier the model config was being ignored during patching which might cause issues like initializing with wrong
max_position_embeddingsHow
Gets required args from model_config and passes it to patching functions.