[New Feature][Roadmap requested]Do you have a roadmap right now?

OptimalScale / LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

https://optimalscale.github.io/LMFlow/

Apache License 2.0

8.21k stars 818 forks source link

[New Feature][Roadmap requested]Do you have a roadmap right now? #605

Open cauwulixuan opened 1 year ago

cauwulixuan commented 1 year ago

Is your feature request related to a problem? Please describe. I am very interested in this project, you guys did a great job. And I wonder if there is a roadmap provided? For instance:

Is there a plan to implement the following functionalities in this project: model compression, quantization, and pruning
Is there a plan to support other fine-tuning algorithms, such as QLORA、freeze、p-tuning? ...

Describe the solution you'd like A clear and concise roadmap should be helpful.

TensorBlast commented 1 year ago

I have created a PR for support for QLoRA (Added QLoRA support for Decoder transformers with tune_strategy "Normal" #613)

Once it is reviewed hopefully it get's merged.

shizhediao commented 1 year ago

Hi, Here are some important features we are going to implement:

Efficient fine-tuning: e.g., p-tuning, and other adapters
Efficient inference: e.g., speculative decoding, vllm
Tool and Agent: e.g., ToolLLM
Model compression: quantization, distillation, pruning

As for the timeline, we are working on Efficient inference and Model compression right now and plan to focus on fine-tuning and tool in Octomber.

Welcome to contribute to LMFlow! Thanks!