How to convert LoRA adapters for using?

[sidharthiimc]

from llm_rs import Llama

model = Llama("path/to/model.bin", lora_paths=["path/to/lora.bin"])

Need way to convert lora adapters as well

[LLukas22]

At present, you must manually convert your LoRa weights through a script like this: convert-lora-to-ggml.py

With that said, it's important to note that LoRa support is likely limited to LLama-based models, as the LoRa-GGML format has not been standardized yet.

Could you please specify the architecture for which you are trying to create a LoRa?

[sidharthiimc]

I was using llama. And found performance not upto mark with 7B. I will stick to 8bits usage.