search

VinAIResearch / PhoGPT

PhoGPT: Generative Pre-training for Vietnamese (2023)
Apache License 2.0
739 stars 67 forks source link

[Question] Hardware requirements with vLLM #13

Closed hodacthanhminh closed 9 months ago

hodacthanhminh commented 9 months ago

To fully fine-tune vinai/PhoGPT-7B5 or vinai/PhoGPT-7B5-Instruct on a single GPU A100 with 40GB memory, it is advisable to employ the decoupled_lionw optimizer with a device_train_microbatch_size set to 1.

How about minimum and recommended hardware requirements when fine-tune vinai/PhoGPT-7B5 with vLLM?

datquocnguyen commented 9 months ago

vLLM is an inference framework only.