How can I merge the LoRA weights into the base model?

[pantDevesh]

Is there a script for this?

[mkserge]

You can do something like this

from mistral_inference.model import Transformer
model = Transformer.from_folder(args.model_path, device=f"cuda:0")
model.load_lora("/path/to/lora.safetensors", device=f"cuda:0")
safetensors.torch.save_model(model, "/path/to/merged.safetensors")

[forest520]

How to perform inference with a LoRA model using Python code, if save_adapters = True?