search

PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
11.99k stars 2.93k forks source link

implemention of lqlora #8607

Closed Liebele closed 2 months ago

Liebele commented 3 months ago

PR types

PR changes

Description

paddle-bot[bot] commented 3 months ago

Thanks for your contribution!

CLAassistant commented 3 months ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Liebele commented 3 months ago

使用paddle中LoRA支持的量化算法进行混合量化。 使用LQ-LoRA，需要先获取整数线性规划的准备数据，接着使用整数线性规划求解每个矩阵对应的量化算法，然后进行迭代初始化获取修改后的原模型及LoRA模块的参数。 在使用LQ-LoRA进行微调时，将“weight_quantize_algo”设置为“lqlora”，同时提供“qconfig_path”以及“lqlora_state_dict_path”（分别对应每个矩阵对应的量化算法、迭代初始化后的原模型及LoRA模块的参数）。

Liebele commented 3 months ago

使用该方法在LLaMA-7B上的实验结果： image image