search

Coobiw / MPP-LLaVA

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.
382 stars 20 forks source link

请问是否支持 流水线并行 推理 #15

Closed valencebond closed 5 months ago

valencebond commented 7 months ago

从而减少 bubble time

Coobiw commented 6 months ago

推理上应该不太存在bubble time的问题，这个主要是训练上的

现在推理没支持PP，但是支持了mapping到multi-gpu上（采用了transformers的device_map="auto"的方法实现）

可以参考刚才的commit：https://github.com/Coobiw/MiniGPT4Qwen/commit/683326bcba26ca6dfd516f5ef48f2c35e71b2c0d

Coobiw commented 5 months ago

It's been a long time. I'll close this issue.