FMInference / FlexLLMGen

Running large language models on a single GPU for throughput-oriented scenarios.
Apache License 2.0
9.2k stars 547 forks source link

Could flexgen be used for training? #113

Open leiwen83 opened 1 year ago

leiwen83 commented 1 year ago

Hi,

As I understand, current flexgen only serve as infer llm in small gpu, which would not happen if without flexgen. As llm training is also resource monster process, could flexgen be applied in the training process? So that with limited cpu memory and gpu could train big model?