Do we have some inference accelerate method for new llava-next-video models?

LLaVA-VL / LLaVA-NeXT

Apache License 2.0

2.4k stars 167 forks source link

Do we have some inference accelerate method for new llava-next-video models? #105

Open LetheRiver0 opened 2 months ago

LetheRiver0 commented 2 months ago

Hi, Amazing job for new llava-next-video model! Since it has 34B params and maybe need more than 1 GPU, so do we have support some inference accelerate method for new llava-next-video models? like sglang deploy. Thanks~

HaiyiMei commented 1 month ago