foundation-model-stack / fms-hf-tuning

🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.
Apache License 2.0
9 stars 30 forks source link

remove merge model for lora tuned adapters #197

Closed anhuong closed 2 weeks ago

anhuong commented 2 weeks ago

Description of the change

Before we needed to merge the LoRA adapter in order for TGIS to properly load the model. Now that this feature is available in vLLM, we can remove the merging of the LoRA adapter.

Related issue number

How to verify the PR

Added unit tests to verify the behavior.

Was the PR tested