replit / ReplitLM

Inference code and configs for the ReplitLM model family

https://huggingface.co/replit

Apache License 2.0

925 stars 80 forks source link

Convert ReplitLM to MPT #15

Closed madhavatreplit closed 1 year ago

madhavatreplit commented 1 year ago

Why

We want to convert to ReplitLM to MPT to make it compatible with Mosaic's LLM Foundry for further training and finetuning.

What changed

changed the model code files
converted and exported the model checkpoint weights in a format compatible with MPT

Testing

Can run training and finetuning with LLMFoundry.
Can load models with transformers

Rollout

This is not backward compatible anymore due to model class changing.

[] This is fully backward and forward compatible