How much GPU memory is needed to run the demo

NJU-LHRS / LHRS-Bot

VGI-Enhanced multimodal large language model for remote sensing images.

Apache License 2.0

81 stars 7 forks source link

How much GPU memory is needed to run the demo #7

Closed Nevaeh7 closed 3 months ago

pUmpKin-Co commented 4 months ago

18GB of RAM suffices for loading processes, while 20GB of RAM is considered safe for serving and engaging in conversations (loading with bf16 or fp16).

Nevaeh7 commented 3 months ago

18GB RAM 足以用于加载进程，而 20GB RAM 对于服务和参与对话被认为是安全的（使用 bf16 或 fp16 加载）。

Thanks your reply