search

ali-vilab / Ranni

https://ranni-t2i.github.io/Ranni/
Apache License 2.0
213 stars 15 forks source link

Inference GPU memory #12

Open wf1024966 opened 6 months ago

wf1024966 commented 6 months ago

Hi, I want to know, What is the minimum GPU memory required? I had inferenced the model in GPU RTX3090 24G, but it turned out of CUDA out of memory, what can I do to reduce the GPU memory ? Thanks.

thss15fyt commented 6 months ago

The GPU memory usage is about 38GB. It is noted that the two main components are running individually. To save memory, you could move the Llama and stable diffusion model to cpu first, and move them to gpu when using them. For applying text-to-panel, move the Llama to gpu. For applying panel-to-text, move the stable diffusion to gpu.