search

openvinotoolkit / training_extensions

Train, Evaluate, Optimize, Deploy Computer Vision Models via OpenVINO™
https://openvinotoolkit.github.io/training_extensions/
Apache License 2.0
1.14k stars 443 forks source link

LoRA or QLoRA for LLMs? #4084

Open epage480 opened 3 weeks ago

epage480 commented 3 weeks ago

I realize OpenVINO was originally made for vision models but I'm interested in using OpenVINO for fine-tuning LLMs. It appears there is support to fine-tune for ViT models but not for language models.

Are there any plans to add LoRA fine-tuning functionality for LLMs or is there a way to adapt a LLM to OpenVINO's constraints?

harimkang commented 3 weeks ago

Hi @epage480, We are also very interested in extending the Language models, but there is no timeline yet. (it's because we're focused on Vision first) Fortunately, the VLM models (CLIP, etc.) will be added in the near future. so i think we'll see an extension to languages in the future. If I have any updates on this, I'll share them here.