Closed TejasRavichandran1995 closed 5 months ago
MNN-2.9.0 will support apply lora on device. But now there are some accuracy problems caused by quantization.
Sure. Thanks @wangzhaode . Any planned rough timelines on the 2.9.0 release?
Marking as stale. No activity in 30 days.
the llm-export utility https://github.com/wangzhaode/llm-export seems to have support to directly export a lora.mnn file during conversion in the llm_export.py .
However , it seems to me the framework does not yet support inference with lora.mnn exported file. Any pointers regarding this would be useful :). @wangzhaode